Senior ML OPs Engineer

Overview

Senior MLOps Engineer (Applied AI Focus)

Location:

Remote (hybrid available)

Pay: 198K
-230K

* Unfortuantely we are unable to provide sponsorship at this time

A rapidly growing technology company is building the operating system for creator‑ and content‑driven growth. The team is driven by a mission to make businesses more human and empower individuals to have greater impact. The culture centers around intentionality, collaboration, continuous improvement, and being a genuinely good human in day‑to‑day work.

The organization has been consistently recognized for excellence across workplace culture, innovation, and product leadership. Team members enjoy a flexible work model that blends in‑person collaboration with remote flexibility, encouraging creativity, connection, and individual work‑style preferences.

We are looking for passionate, forward‑thinking builders to join our journey and help shape the future of our industry.

As a Senior MLOps Engineer on the Product Innovations team, you will serve as the technical lead for Applied MLOps. Your work will bridge the gap between experimental research and production‑ready AI systems, with a strong emphasis on ground‑truth generation, model evaluation, and the pre‑/post‑processing infrastructure that powers large‑scale vector embeddings and applied AI features.

You will play a pivotal role in defining best practices, driving evaluation frameworks, and building the infrastructure that enables experimentation and productionization at scale.

• Architect Annotation & Measurement Pipelines
• Design and implement human‑in‑the‑loop and automated annotation workflows.
• Build systems for reliable quality metrics, confidence scoring, and inter‑annotator agreement (IAA).
• Stand up annotation tools and processes to support ground‑truth generation at scale.
• Drive Cost‑Efficient Model Evaluation
• Create ground‑truth datasets, golden sets, and model‑by‑model evaluation criteria.
• Benchmark model performance to guide data science decisions and optimize compute and cost efficiency.
• Directly influence product margins by helping deploy the right model for the right use case.
• Enforce Applied MLOps Standards
• Implement frameworks for deterministic preprocessing, PII scrubbing, and model‑as‑a‑judge evaluation loops.
• Establish robust production standards for safety, consistency, and reliability.
• Collaborate on Model & Product Strategy
• Partner closely with Data Science to drive applied generative AI decisions.
• Influence model selection, architecture, and optimization approaches.
• Build & Integrate with Infrastructure
• Work with engineering teams to integrate measurement and evaluation loops into the broader cloud ecosystem (AWS/GCP).
• Ensure automation, observability, and smooth model lifecycle operations.

• Annotation & Evaluation Expertise
  • Experience building and managing annotation workflows using commercial or custom tools.
  • Strong understanding of quality metrics, ground‑truth processes, and IAA methodologies.
• Applied MLOps Practitioner
  • Hands‑on experience with model monitoring, versioning, evaluation frameworks, and production ML infrastructure.
  • Pragmatic and execution‑focused—prioritizing scalable solutions over theoretical perfection.
• Technical Proficiency
  • Strong Python skills and experience building integrations, pipelines, and tooling for ML systems.
  • Understanding of performance trade‑offs between model size, cost, latency, and accuracy.
• Cloud & Infrastructure Fluency
  • Experience working within cloud ML environments (AWS, GCP, or equivalent).
  • Ability to collaborate effectively with Dev Ops/SRE teams.
• Collaborative Technical Leader
  • Comfortable acting as a force multiplier for data science teams.
  • Strong communication skills and a cross‑functional mindset.

Note: C…