AI Data Engineer

Veeva Systems is a mission-driven organization and pioneer in industry cloud, helping life sciences companies bring therapies to patients faster. As one of the fastest-growing SaaS companies in history, we surpassed $2B in revenue in our last fiscal year with extensive growth potential ahead. At the heart of Veeva are our values:
Do the Right Thing, Customer Success, Employee Success, and Speed. We’re not just any public company – we made history in 2021 by becoming a public benefit corporation (PBC), legally bound to balancing the interests of customers, employees, society, and investors. As a Work Anywhere company, we support your flexibility to work from home or in the office, so you can thrive in your ideal environment.

Join us in transforming the life sciences industry, committed to making a positive impact on its customers, employees, and communities.

This role is responsible for ensuring the reliability, accuracy, and safety of our Veeva AI Agents through rigorous evaluation and systematic validation methodologies. We’re looking for experienced candidates with a meticulous, critical mindset, strong analytical problem-solving, excellent communication with technical audiences, and the ability to learn application areas quickly.

• Define and establish comprehensive evaluation strategies for new AI Agents, prioritizing test data integrity and real-world usage coverage.
• Programmatically and manually evaluate LLM-generated content against predefined metrics such as factual accuracy, contextual relevance, coherence, and safety standards.
• Design, curate, and generate diverse, high-quality test data sets, including challenging prompts and scenarios, to proactively identify system biases, unsafe content, hallucinations, and critical edge cases.
• Develop, implement, and maintain scalable automated evaluation pipelines to continuously validate agent behavior and prevent regressions with new features and model updates.
• Understand model behaviors and assist in trace and root-cause analysis of identified defects or performance degradations.
• Clearly document, track, and communicate performance metrics, validation results, and bug status to the broader development and product teams.

• Strong understanding of data quality principles and validation methods for bias, integrity, and quality standards; ability to craft diverse and adversarial test data to uncover AI edge cases.
• Advanced prompt engineering expertise, deep knowledge of LLM common failure modes, and skill in creating evaluation scenarios that test reasoning, action planning, and adherence to system instructions.
• Proficiency in designing and deploying automated evaluation pipelines; familiarity with quality metrics such as task success rate, semantic similarity, and sentiment analysis.
• Comfort debugging agentic systems, including tracing and interpreting an agent’s internal reasoning, tool use, and action sequence to pinpoint failure points.
• Programming proficiency in Python, ability to develop custom evaluation frameworks, write scripts, and integrate pipelines with CI/CD systems; familiarity with standard test automation tools such as Pytest.
• Bachelor’s degree in Data Science, Machine Learning, Computer Science, or a related field with experience in Gen AI / LLMs.
• High work ethic and strong integrity and honesty, aligned with Veeva’s values.
• Applicants must have the unrestricted right to work in the United States or Canada;
  Veeva will not provide sponsorship at this time.

Veeva is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, sex, sexual orientation, gender identity or expression, religion, national origin or ancestry, age, disability, marital status, pregnancy, protected veteran status, protected genetic information, political affiliation, or any other characteristics protected by local laws, regulations, or ordinances. If you need assistance or accommodation due to a disability or special need when applying for a role or in our recruitment process, please contact us at ta