LLMOps Engineer

LLMOps Engineer

The Large Language Model Ops (LLMOps) Engineer ensures Brooklyn Sports & Entertainment's AI systems are safe to run in production and safe to evolve over time. This role builds and operates the infrastructure that powers the company's AI platform, including deployment pipelines, evaluation systems, observability tooling, and infrastructure for agent-based workflows.

Working primarily within AWS and Amazon Bedrock, the LLMOps Engineer builds the production environment required to reliably operate AI systems that support ticketing operations, venue analytics, digital fan engagement, partnerships workflows, and internal engineering tools.

The role works closely with the AI Engineer, sharing responsibility for production operations while focusing on infrastructure reliability and AI platform governance.

This role reports into the VP, Artificial Intelligence.

AI Infrastructure and Platform Reliability

• Design and operate the AWS-based infrastructure foundation for AI and agent systems.
• Manage integrations with Amazon Bedrock and other model providers.
• Implement secure environment configuration, secrets management, and infrastructure policies.
• Build scalable systems capable of supporting ongoing operations.

AI Development Pipeline

• Build and maintain CI/CD pipelines for AI systems, including deployment of prompts, models, and agent workflows.
• Implement versioning for models, prompts, and Lang Graph agent graphs.
• Design environment promotion standards and automated rollback mechanisms.

CodeAI Development Pipeline

• Design and operate the CodeAI development pipeline, enabling AI agents to assist with software development tasks.
• Build sandboxed environments allowing AI agents to safely interact with repositories, CI systems, and development workflows.
• Establish governance and security guardrails for AI-assisted development and vibe coding practices.

Observability and Evaluation

• Implement monitoring systems using AWS Cloud Watch and evaluation dashboards.
• Define Service Level Objectives (SLOs) for AI system reliability.
• Build shared evaluation infrastructure that measures hallucination rates, tool-call success, and workflow reliability.
• Monitor and optimize cost-per-request metrics.

Shared Production Ownership

• Participate in deployment, monitoring, and incident response for AI systems.
• Develop operational playbooks that reduce mean time to recovery and improve system resilience.

WHAT YOU WILL BRING

• 3–6 years of experience working in Dev Ops, MLOps, or infrastructure engineering roles.
• Strong experience operating AWS cloud infrastructure.
• Experience deploying and operating AI or machine learning systems in production environments.
• Experience with Amazon Bedrock or similar LLM platforms.
• Experience building CI/CD pipelines and automated deployment systems.
• Experience implementing observability systems and operational dashboards.

WHO YOU ARE

• You enjoy building systems that make complex technology safe and reliable to operate.
• You are motivated by operational excellence and reliability engineering.
• You demonstrate a strong growth mindset and actively learn new technologies as the AI ecosystem evolves.
• You enjoy experimenting with new technologies while maintaining production-quality standards.
• You collaborate closely with engineers building AI applications to ensure systems scale safely.

COMPENSATION

$130,000 - $150,000 annual salary.

Full-time employees are eligible for a robust slate of total rewards, including:

• Bonus eligibility
• Medical, dental, and vision coverage; HSA and FSA eligibility
• 401k Employer Match at 4%
• Competitive PTO policy & Company Holidays
• Parental leave policy eligible after 6 months of service
• Access to events at Barclays Center, subject to ticket availability
• Free lunch onsite Monday - Thursday; onsite barista bar
• And more!

WORK ENVIRONMENT

Works primarily in an office environment.

We are an Equal Employment Opportunity ("EEO") Employer. It has been and will continue to be a fundamental policy of the Company not to discriminate on the basis of race, gender, gender identity, transgender status, pregnancy and lactation accommodations, marital status, partnership status, domestic violence victim status, sexual orientation, age, national origin, alienage, immigration, or citizenship status, veteran or military status, disability, genetic information, height and weight, arrest or conviction record, caregiver status, credit history, unemployment status, sexual and reproductive health decisions, salary history, status as a victim of domestic violence, stalking, and sex offenses, or any other characteristic prohibited by federal, state or local laws.