AI Engineer V; Principal

The AI Engineer V (Principal) role at Assist Rx, LLC is the most senior individual-contributor technical authority for AI and machine learning. This position defines and drives enterprise AI strategy while leading the design, build, and deployment of production-grade AI solutions, with a focus on safe and compliant use of AI in a specialty medication access ecosystem that involves protected health information.

In partnership with senior leadership across Engineering, Product, and Architecture, this role sets enterprise AI architecture, evaluation standards, and responsible AI practices. The Principal engineer leads complex AI solution delivery spanning generative AI, predictive modeling, and intelligent document processing, ensuring systems meet reliability, quality, and compliance expectations.

• Define the multi-year AI technical strategy and roadmap in partnership with the Vice President, Engineering, Product, and Architecture, aligning AI investments to business value, feasibility, and risk.
• Lead design of enterprise-scale AI and machine learning solutions, including generative AI, predictive models, and intelligent document processing across platforms, products, and clients.
• Architect large language model applications using retrieval augmented generation, prompt engineering, structured outputs, function and tool calling, and agent-based orchestration.
• Set patterns for agentic workflows, including task decomposition, tool and function calling, memory and state management, routing, human review checkpoints, boundaries, and fallback paths for high-impact decisions such as benefits verification and prior authorization.
• Identify new AI opportunities and emerging technologies, and lead proofs of concept that transition proven pilots to scale.
• Determine the right approach per problem, including prompting, retrieval augmentation, tool use, task-specific models, and deterministic logic when models are not appropriate.
• Set the standard for AI feature design documentation, covering intended behavior, context and retrieval design, prompt and agent structure, evaluation approach, and fallback behavior.
• Own the enterprise AI reference architecture, platform strategy, and engineering standards, including model access, prompt and version management, and retrieval services, and ensure adoption across engineering teams.
• Lead evaluation of commercial models, AI platforms, and vendor tools, and recommend build versus buy to executive leadership based on quality, cost, security, strategic fit, and time to value.
• Guide vendor and cloud provider relationships technically, including model selection, capacity, pricing, and risk.
• Anticipate scaling, reliability, and cost challenges, and drive architectural investments to address them before client impact.
• Set organization-wide targets for accuracy, groundedness, hallucination rate, task completion, latency, and cost, guiding model selection, tuning, and evaluation.
• Define evaluation strategy, benchmark datasets, and automated test frameworks to measure output quality and detect regressions before release.
• Optimize retrieval pipelines, embeddings, chunking strategies, and inference performance to improve quality and reduce cost per transaction at scale.
• Direct error analysis and human-in-the-loop review programs with operations and clinical subject matter experts, ensuring improvements flow into prompts, models, and guardrails.
• Establish experimentation practices including A/B testing across prompts, retrieval configurations, model selection, and workflow design.
• Define and enforce quality gates and regression thresholds so AI features meet defined accuracy and safety criteria before release.
• Set technical direction for pipelines that prepare structured and unstructured healthcare data (such as enrollment forms, payer documents, and clinical notes) for training, retrieval, and inference.
• Partner with Data Engineering and Architecture to define enterprise data contracts, feature sets, vector stores, and lineage for AI workloads.
• Define enterprise standards for de-identification, masking, and data minimization when working with protected health information.
• Define enterprise standards for CI/CD, source control, containerization, and infrastructure-as-code practices for AI workloads.
• Establish versioning, promotion, and rollback practices for prompts, agents, and model configurations as versioned artifacts alongside application code.
• Set observability strategy for accuracy drift,…