Machine Learning Engineer

Position: Staff Machine Learning Engineer

Staff Machine Learning Engineer

Employment Type

Location Type Overview Application

Location:

Remote (US) Pay: $230,000–$300,000 + bonus + equity A late-stage, venture-backed enterprise AI company is hiring a Staff ML Engineer to set the technical direction for its agent systems. This is model and systems work that ships to large enterprise customers, not a thin layer over someone else's API. What you'll do Design multi-step LLM systems: reasoning, retrieval, planning and tool use Own evaluation,

Relevance is based on skills and concepts, not a guarantee that you qualify.

Covered skills: AI fundamentals Machine learning Deep learning GenAI Cloud LLM evaluation

Skills to close:
None listed

Build the skills for this job

Coming-soon courses matched to this posting: outline, waitlist, and skill path.

1. Defining Evaluation Scope and Use Cases

Map product requirements to testable behaviours and identify what good looks like for your specific LLM application.

2. Building Representative Test Sets

Create test datasets that capture real-world diversity, edge cases, and adversarial inputs for your domain.

3. Writing Scoring Rubrics and Metrics

Define explicit, measurable criteria for correctness, safety, faithfulness, and task‑specific quality.

4. Implementing LLM-as-Judge Pipelines

Build scalable automated evaluation systems using LLMs as judges, with calibration and quality controls.

5. Evaluating RAG Systems End to End

Separate retrieval quality from generation quality and measure faithfulness, citation, and refusal behavior.

6. Running Error Analysis and Regression Testing

Turn evaluation results into insights and build safety nets that catch regressions when systems change.

7. Evaluation in Deployment and Production

Extend evaluation to live environments, monitor drift, and adapt to device, latency, and user context constraints.

This job posting may span more than one career level. In addition to base salary, this job is eligible to receive equity in the form of res…