SRE​/LLM Ops Engineer

Overview

Company Description
At CluePoints, we’re redefining how clinical trials are run. As the premier provider of Risk-Based Quality Management (RBQM) and Data Quality Oversight software, we harness advanced statistics, artificial intelligence, and machine learning to ensure the quality, accuracy, and integrity of clinical trial data, helping life sciences organisations bring safer, more effective treatments to patients faster. We’re proud to be an ambitious, fast-growing technology scale-up with a dynamic and diverse international team representing more than 40 nationalities.

Collaboration, flexibility, and continuous learning are part of our DNA. Guided by our values of Care, Passion, and Smart Disruption, we’re united by a shared mission: to create smarter ways to run efficient clinical trials and deliver AI-powered insights that improve human outcomes worldwide.

The SRE, LLMOps (AI Platform) ensures our LLM-powered services are reliable, observable, and safe in production on Azure and Kubernetes. You’ll blend classic SRE disciplines with LLM-specific operations: robust evaluation pipelines, prompt/version governance, model/vendor failover, guardrails, and cost/performance monitoring. You know how to build automation with Lang Chain/Lang Graph, operate API-based LLMs in production, and manage the inherent non-determinism of models through rigorous testing and observability.

• Experience:
  
  5+ years in SRE/Dev Ops/Platform Engineering with 1–2+ years operating LLM or ML-backed applications in production (API-based or hosted models).
• LLMOps: hands-on with Lang Chain/Lang Graph building end-to-end chains/agents and RAG flows; comfort with vector stores (e.g., Azure AI Search, Pinecone), prompt/version control, and dataset tooling.
• Observability: proficiency instrumenting LLM traces and app telemetry, alert tuning, and root-cause analysis; familiarity with Lang Smith and/or Arize Phoenix (token/cost tracking, latency, failure modes).
• Cloud & platform: strong Azure and Kubernetes (AKS) background;
  Git Ops (Flux/ArgoCD), Helm/Kustomize; CI/CD (Git Hub Actions/Git Lab/Jenkins);
  IaC (Terraform); secrets, networking, and security baselines.
• Languages & tooling:
  Python (preferred) and one of Type Script/Go; REST/Graph
  
  QL;
  OpenAI/Azure OpenAI/Anthropic APIs; experience with Redis caches, message queues, and feature flags.

• Instrument deep observability: implement tracing for LLM chains/agents (inputs, outputs, token usage, latency, model/version), correlate with app metrics/logs, and set actionable alerts; leverage Lang Smith/Arize Phoenix (or similar) and Open Telemetry where appropriate.
• Safety & guardrails: integrate content safety, PII redaction, jailbreak/prompt-injection defenses, and policy-based rails; document exceptions and reviewer workflows. Prefer native platform features (e.g., Azure AI Content Safety) or programmable rails (e.g., NVIDIA NeMo Guardrails).
• Cost & capacity management: monitor token and request costs, throughput, and rate limits; implement caching, request shaping, and multi-tier model selection to balance quality, latency, and spend.
• Build evaluation & testing pipelines: create golden datasets and automated evals (offline + CI/CD + canary) to catch regressions from code, prompt, data, or model changes; use Lang Smith/OpenAI Evals (or equivalents) to track quality trends over time.
• Platform operations on Azure/Kubernetes: ensure secure, compliant, and cost-efficient operation; maintain IaC, secrets, networking, scaling, and DR/BCP; partner with Security and QA on regulated SaaS controls.
• Cross-functional enablement: work with product/dev teams to set acceptance criteria for AI features, add runtime feature flags/kill-switches, and embed evals/telemetry from day one.

• Private Medical Insurance through Vitality Health (full hospital cover, 24/7 GP, and therapy sessions)
• Group Critical Illness Cover with Aviva
• Life Insurance (death-in-service lump sum)
• Pension Scheme with 9% employer contribution via Scottish Widows
• Opportunities for professional development and sponsored certifications
• A hub-based hybrid model that blends flexibility…