Lead Software Engineer - Quality & AI

* To get the best candidate experience, please consider applying for a maximum of 3 roles within 12 months to ensure you are not duplicating efforts.
* Job Category Software Engineering Job Details
**** About Salesforce
**** Salesforce is the #1 AI CRM, where humans with agents drive customer success together. Here, ambition meets action. Tech meets trust. And innovation isn’t a buzzword — it’s a way of life. The world of work as we know it is changing and we're looking for Trailblazers who are passionate about bettering business and the world through AI, driving innovation, and keeping Salesforce's core values at the heart of it all.

We are seeking an experienced Salesforce Software Quality Lead Member of Technical Staff (LMTS) with a strong technical background. This engineer should have an AI first approach and be able to help the team translate from traditional testing to using AI in driving acceleration and accuracy in testing and be a multiplier using AI tools. Prior experience as a member of a Data Science, ML Science, or ML Engineering team is a plus.

As a Quality LMTS, you will play a key role in defining, implementing, and maintaining the quality strategy for Salesforce products, with a strong focus on integrating cutting-edge AI technologies. You will collaborate with cross-functional teams, including product development, data science, and AI teams, to ensure that Salesforce’s AI-driven solutions meet the highest quality standards for enterprise applications.## This role requires onsite presence in San Francisco, California office.
*
* Key Responsibilities:

**** Drive Innovation & Adoption:
** Identify opportunities for integrating AI/ML solutions into existing workflows, propose new ones that are high-quality and efficient, and generally positively influence engineering decision-making by communicating and evaluating solution options, and facilitating agreement among key stakeholders. Ensure that we are continuously raising our standard of engineering excellence and producing high business impact.
** Continued Excellence in AI/ML:
** Be tuned into developments in the field of AI/ML, by tracking publicly available models, frameworks, libraries and publications, while being focused on their practical application and relevance to Salesforce.
** Technical Leadership on Proof of Concepts:
** Influence the direction of R&D as a whole through your technical, process, or product knowledge leadership. Industry-expert level in understanding of quality concerns and validation techniques. Able to drive behaviors and initiatives that both focus on quality and throughput.
** Collaborate on AI Model Improvements:
** Work closely with data scientists and machine learning engineers to provide feedback on LLM performance and recommend improvements based on quality testing results.
** Evaluate LLM Performance:
** Assess and optimize Large Language Models (LLMs) for quality, accuracy, relevance, and safety in various use cases.  Human Evaluation:
Oversee human evaluation processes for subjective quality dimensions, such as response engagement, user safety, and contextual accuracy.
** Define Quality Strategies:
** Develop and implement robust quality strategies for Salesforce’s SaaS products, focusing on AI/ML and LLM integrations.
** Multiplier:
** Provide technical leadership for critical areas that significantly impact customer success. You have depth of expertise in key technologies and you are often consulted on the design and delivery of new solutions. You bring new best practices to R&D and actively ensure that they are being used.

Continues to deepen and widen their understanding of the application and drives discussion of sometimes complex or controversial issues in open forums such as concept reviews, VAT discussions, and cross team discussions.
** Cross-Platform

Collaboration:

** Use your understanding of customers’ needs across industries and multiple technology landscapes (CRM, Modern Data Stack, Analytics & BI, CRM and AI) to develop solutions across Salesforce's technology stack. Work in a consultative fashion to improve communication, teamwork and alignment among teams inside and outside of the…