AI Safety & Evaluation Engineer

Title: AI Safety and Evaluations Engineer
Job Type: Contract
Contract Length: 12 Months
Pay Range: $50/hr – $175/hr
Start Date: ASAP
Location: Remote

About the Opportunity: Our client, a leader in AI testing and Generative AI solutions, is looking for a skilled AI Safety and Evaluations Engineer to join their team for a 12-month engagement. This project involves designing and building rigorous evaluation frameworks to measure model bias, hallucinations, and toxicity, ensuring models are safe and compliant before deployment. This is a high-impact role that requires a self-motivated professional who can hit the ground running and deliver results quickly.

• Designing and building rigorous evaluation frameworks to measure model bias, hallucinations, and toxicity.
• Creating automated "Eval" datasets to benchmark new models before they are promoted to production.
• Developing metrics for "Grounding" and "Faithfulness" in RAG-based systems.
• Building monitoring tools that flag harmful or non-compliant AI outputs in real-time.
• Partnering with legal and ethics teams to translate policy into technical safety constraints.

• 3+ years of experience in AI Research or Quality Engineering.
• Deep expertise in model evaluation techniques and NLP metrics (ROUGE, BLEU, BERTScore). This isn't a learning role—you need to be a subject matter expert.
• Demonstrated ability to work autonomously and manage your own time effectively to meet project goals.
• Experience with Python, data analysis tools, and LLM-as-a-Judge frameworks.
• Strong communication skills to provide clear and concise status updates to the project team.