Lead Specialist, AI Scientist

We are seeking a Lead Specialist, Applied AI Systems Engineer , to design, build, and continuously improve the intelligent systems that support Pearson’s work developing the next generation of K–12 assessment and learning.

This role sits within the Measurement & Learning team and operates at the intersection of Content Development, Learning Science, Technology, AI Science, and Psychometrics, partnering closely with software engineers and the AI Data & Knowledge Architect to turn assessment and measurement expertise into testable AI capabilities within our assessment authoring system, which supports formative and summative assessment for Pearson and its partners.

This hands‑on role joins in the early phases of this work, improving and extending complete AI systems built on a shared knowledge foundation, rather than focusing on prompts or chatbots. The work spans predictive machine learning, statistical methods, retrieval, graph‑based approaches, generative and agentic AI, and deterministic software, chosen for reliability rather than novelty.

We are particularly interested in innovators who have built AI or machine‑learning systems whose outputs must be rigorously evaluated against ground truth, expert judgment, or reliable evidence, such as scientific, healthcare, financial, cybersecurity, legal, or assessment applications. Candidates whose experience is mainly in conversational or customer‑service AI should also show substantial experience evaluating systems against defined performance standards. Experience in education, learning, or assessment is strongly preferred when it involves the systems beneath products, such as domain and skill models, learner models, standards and competency frameworks as structured data, or adaptive and recommendation engines, rather than standalone tutoring, chatbot, or content‑generation applications.

End-to-End

AI Systems Design

• Design complete, modular AI systems spanning knowledge and data inputs, retrieval, models, agents, tools, deterministic services, human review, evaluation, and integration
• Break complex problems into the right combination of predictive ML, statistical methods, generative models, retrieval, graph reasoning, deterministic software, and human decision points
• Build working systems and reference implementations, and partner with software engineers to bring them into the assessment authoring system
• Design architectures that can evolve as models, data, domain knowledge, and evaluation evidence change

Agentic & Knowledge‑Grounded AI

• Design and implement agentic and multi‑step workflows using models, tools, retrieval, state, structured outputs, and human decision points
• Build knowledge‑grounded systems that use graph, semantic, vector, and hybrid retrieval as appropriate
• Co‑design knowledge and graph architecture with the AI Data & Knowledge Architect, contributing substantively rather than treating the knowledge layer as a black box
• Develop AI capabilities for content creation and revision, verification, alignment across frameworks, guidance, and analytic insights

Evaluation‑Driven Development

• Make evaluation part of how systems are built, not a final step
• Build and maintain evaluation benches, including expert‑adjudicated datasets, challenge sets, regression suites, and protected holdout data
• Partner with psychometricians and content experts to turn domain expectations into measurable criteria
• Analyze failure modes, critical errors, and performance across meaningful data and task slices, not only aggregate metrics
• Monitor deployed systems for drift and emerging failures, and use findings to drive improvement
• Develop predictive models, classifiers, and ranking methods where they outperform generative AI
• Evaluate tradeoffs among prompting, retrieval,…