LLM Ops Engineer

LLM Ops Engineer page is loaded## LLM Ops Engineer locations:
PHL-Taguig City-Citi Plaza:
Nottingham, United Kingdom time type:
Full time posted on:
Posted Todayjob requisition :
R0111794
** About the Role
** We are seeking a skilled and proactive
** LLM Ops Engineer
** with expertise in
** Large Language Model (LLM) Operations
** to join our growing AI and software development team. This role is critical in ensuring the reliability, performance, and evolution of LLMs and data pipelines in production environments.

You’ll understand the roles of Platform, AI Engineering, Governance and Product. We have built a technical stack that can be expressed as three main categories; tracing, Annotations and experiments with plans to build a Data Science Workspace.  We are looking for an engineer who can help us maintain the pipelines, experimentation platform and data science workspace.  Ideally you’ll have experience with Langfuse (or similar tools) and how data science work spaces.

The LLM Ops team is a new team and we are building out the use case, so you’ll be part of a team that is pushing on new frontiers in the area of LLM Ops.  You’ll be working with other LLM Ops engineers, AI engineers, data scientists, operations personnel and our product manager.
** Responsibilities**
* ** Data & System Integration**  + You’ll be responsible for configuring the platform, integration.  You’ll understand the roles of Platform, AI Engineering, Governance and Product.  You’ll be responsible for completion of the LLM Ops stack, including Platform (DSQ, Langfuse), Langfuse integrations for AI components, LLM Ops workflows, help with data annotations, basic monitoring and analytics.  + Work with data  scientists and engineers to ensure high-quality, real-time data feeds into LLM systems.  

+ Build and maintain observability tools for data quality, model behaviour, and system health.
* ** Model Lifecycle Management**  + Manage the deployment, versioning, and rollback of LLMs in production.  + Help to coordinate and execute model upgrades, ensuring minimal disruption to services.  Supported by data scientists.  + Implement CI/CD pipelines for model deployment and testing.
* ** LLM Monitoring & Maintenance**  + Help to monitor LLM performance in production, including latency, accuracy, and user feedback.  + Help to detect and address LLM drift and data drift using automated tools and custom metrics.  + Collaborate with data scientists to define and refine model evaluation criteria.
** Required Skills & Experience
*** Proven experience in LLM Ops, MLOps, or AI infrastructure roles.
* Strong understanding of LLM behaviour, fine-tuning, and prompt engineering.
* Experience with monitoring tools (e.g., Prometheus, Grafana, MLflow, Evidently AI).
* Experience in AWS Bedrock LLM models
* Proficiency in Python and familiarity with ML frameworks (e.g., Hugging Face Transformers, Lang Chain).
* Experience with cloud platforms (AWS, Azure, or GCP) and containerization (Docker, Kubernetes).
* Experience in Model Optimisation and familiarity with data drift detection, model drift analysis, and automated retraining pipelines.
* Excellent communication and collaboration skills.
** Desired Skills & Experience
*** Experience working in agile software development environments.
* Knowledge of data privacy, model interpretability, and responsible AI practices.
* Contributions to open-source LLM or MLOps projects.
* Experience in Finance sector and/or risk management
** Career Stage:
** Senior Associate
** London Stock Exchange Group (LSEG) Information:
** Join us and be part of a team that values innovation, quality, and continuous improvement. If you're ready to take your career to the next level and make a significant impact, we'd love to hear from you.

LSEG is a leading global financial markets infrastructure and data provider. Our purpose is driving financial stability, empowering economies and enabling customers to create sustainable growth.

Our purpose is the foundation on which our culture is built. Our values of
** Integrity, Partnership**,
** Excellence
* * and
** Change
* * underpin our purpose and set the standard for everything we do, every day. They go to the…