Senior Principal AI Engineer

Overview

Serve as the most senior individual-contributor engineer and principal technical authority within the Commercial AI Center of Excellence (CAI CoE). Own the technical vision for AI-as-a-Service (AIaaS) enablement at enterprise scale. Operate as a full-spectrum AI engineer fluent across the entire lifecycle — data, model training and fine-tuning, retrieval, orchestration, evaluation, and production operations — capable of deep work across traditional AI/ML, GenAI product engineering, and software architecture.

Establish reference architectures, paved-road patterns, and enterprise technical standards for agentic orchestration, tool and MCP design, retrieval, model training, and responsible AI across production systems. Lead complex, cross-team initiatives spanning multiple value streams and influence AI designs to align with enterprise standards. Set long-term technical direction while remaining hands-on with critical AI infrastructure and services.

• Set the multi-year technical vision, reference architectures, and enterprise standards for AIaaS across the organization; act as the final technical authority and escalation point for the hardest AI problems.
• Define and own the enterprise model-training strategy across traditional AI/ML and LLMs; personally train, fine-tune (e.g., QLoRA, LoRA, PEFT, and full fine-tuning), and evaluate models when needed.
• Establish standards for data sourcing, cleaning, versioning, storage, and governance (lineage, licensing/consent, and PII); architect large-scale data and feature pipelines.
• Architect the orchestration and abstraction layers that connect LLMs to tools, data, and sub-agents; set standards for MCP servers and tool-surface design and determine when to use specialized sub-agents versus direct tool exposure.
• Design end-to-end retrieval/RAG systems (chunking strategies, embeddings, vector stores, hybrid search, re-ranking, context assembly, memory).
• Own enterprise evaluation, observability, and safety strategy for AI systems, including offline/online evaluation, tracing, red-teaming, guardrails, and responsible-AI and compliance requirements.
• Drive build-versus-buy decisions, model and vendor selection, and long-term architectural bets to position the organization for future AI advances.
• Optimize performance, cost (token and inference economics), scalability, and reliability of AI workloads in partnership with Security, Cloud Platform, and SRE teams.
• Mentor and grow Principal and Staff engineers and raise the AI engineering bar across the organization.

• 15+ years in AI/ML software engineering with demonstrated Senior Principal-level impact delivering production AI at enterprise scale.
• Full lifecycle mastery across both specialist domains: (a) training and fine-tuning traditional AI/ML models and LLMs (including parameter-efficient methods, quantization, distributed training, and rigorous evaluation); and (b) LLM orchestration, agentic systems, tool/MCP design, and retrieval/RAG in production.
• Deep expertise in distributed systems, cloud-native architecture, and large-scale data/feature pipelines.
• Strong command of data management and governance: dataset storage, versioning, lineage, quality, PII handling, and licensing/consent for training data.
• Proven ability to design developer platforms, APIs, reusable SDKs, MCP servers, and multi-agent orchestrations used by many teams.
• Rigorous approach to AI evaluation, observability and tracing, and responsible-AI guardrails.
• Expertise with cloud platforms (Azure strongly preferred) and a track record of optimizing AI workload cost, performance, scalability, and reliability.
• Ability to influence decisions across many teams without direct authority and to mentor Principal- and Staff-level engineers.
• Experience operating in regulated SaaS environments and meeting security and compliance requirements.

• Bachelor’s degree in Computer Science, Engineering, or related discipline; advanced degree preferred. An equivalent combination of education, training, and experience accepted.
• Industry experience in regulated domains (insurance, fintech, or healthcare).
• Experience with vector databases and retrieval optimization at scale.
• Fin Ops for GenAI: experience modeling and optimizing LLM token and inference costs.
• Data science or classical AI background beyond prompt engineering (statistics, feature engineering, model evaluation).
• Contributions to open-source AI tooling, research, patents, or recognized technical thought leadership.
• Strong executive communication and technical storytelling skills.

US Base Salary Range: $ - $. Base pay offered to new hires may vary based on relevant industry, job-related skills and experience, geographic location, and business needs. The range does not encompass the full potential of the role, which allows for further growth and career progression. The role may be eligible for the Vertex Bonus Plan (VOB), a role-specific sales…