Lead AI​/ML Engineer

Our client is a growing technology consultancy focused on helping banks build and deploy secure, production-grade AI solutions.

They are looking for a Lead AI/ML Engineer to help build the intelligence layer behind an AI platform designed specifically for the banking industry.

This is a hands-on engineering role focused on production LLM applications, RAG, retrieval, model serving, tool orchestration, and evaluation. The goal isn't simply to build an impressive AI demo—the goal is to build systems that produce accurate, verifiable, and auditable answers in a highly regulated environment.

You'll work closely with data engineering, platform engineering, and product teams to build AI systems that can operate securely within banking environments, including self-hosted and on-premises deployments.

• Design and build production-grade RAG and retrieval systems that provide verifiable answers with supporting queries, sources, and citations.
• Develop retrieval and verification workflows across structured and unstructured banking data.
• Deploy and operate self-hosted open-weight LLMs and embedding models, including within individual bank environments.
• Build and evolve the organization's standardized RAG architecture and patterns.
• Design and implement an MCP/tooling layer that allows AI systems to securely interact with governed data, metrics, documents, and customer information.
• Build tool/function-calling workflows with strong auditing, access controls, and clear boundaries around what AI systems can access.
• Develop and own an AI evaluation framework covering golden-question regression testing, retrieval quality, groundedness, answer accuracy, and appropriate "I don't know" behavior.
• Make evaluation a formal release gate for production AI systems.
• Implement LLM guardrails including PII redaction, prompt-injection defenses, context controls, and query/row limits.
• Partner with data engineering to ensure AI systems use governed semantic-layer metrics rather than generating unreliable ad hoc SQL.
• Work with platform and security engineering to deploy reliable AI infrastructure in controlled environments.
• Monitor and optimize latency, inference cost, retrieval quality, and model performance across different models and deployment configurations.
• Document model architecture, evaluation methodology, security controls, and guardrails for customer security reviews and audits.
• Provide technical leadership while remaining deeply hands-on with implementation and production troubleshooting.

• 6+ years of software/ML engineering experience, with at least 2–3 years building and shipping production LLM, RAG, NLP, or generative AI systems.
• Proven experience taking AI systems from prototype to production and real users.
• Strong Python skills and solid software engineering fundamentals.
• Hands-on experience building production RAG/retrieval pipelines.
• Strong understanding of LLM evaluation, including groundedness, retrieval quality, regression testing, and hallucination reduction.
• Experience implementing AI guardrails and security controls.
• Experience deploying or operating self-hosted/open-weight models.
• Ability to reason about the trade-offs between model quality, latency, infrastructure, and cost.
• Strong understanding of prompt engineering, tool calling, structured outputs, and LLM application architecture.
• A strong engineering mindset around accuracy and verifiability, rather than simply building compelling AI demos.
• Comfortable working cross-functionally with data, platform, security, and product teams.

The team works across a modern AI/ML stack including:

Languages:

Python

Model Serving: vLLM, Ollama

Infrastructure: NVIDIA GPUs, CUDA, NVIDIA AI Enterprise

Embeddings:
Modern embedding models and vector search

AI Orchestration: MCP, tool/function calling

Structured AI:
Text-to-SQL, semantic layers, Cube, dbt Metric Flow

Evaluation: LLM evaluation frameworks, groundedness testing, retrieval evaluation

You don't need experience with every technology listed. We're particularly interested in engineers who understand the architecture and engineering principles behind production AI systems and can quickly adapt to new tools.

• Build real AI, not prototypes:
  Your work will be deployed into production and used by banking professionals.
• Own the AI architecture:
  You'll have meaningful influence over RAG, model serving, evaluation, tooling, and guardrails.
• Work with self-hosted AI:
  Build and operate LLM infrastructure where…