Evaluations Lead

Primary

Skills:

Generative AI LLM Evaluation Frameworks (advanced), Quality Assurance Methodologies (advanced), Project Management (intermediate), Human-in-the-Loop Systems (advanced), AI Model Assessment (advanced)

Contract Type: W2

Duration: 8 Months

Location:

Oakland CA (#LI-HYBRID)

Pay Range: $55-$60/HR W2 #LP

Job Summary:

This role is critical in ensuring the quality and scalability of Large Language Model outputs for Turbo Tax by leading human evaluation processes. You will be responsible for designing and overseeing these evaluations to directly improve AI model performance and customer experiences. Your work will transform raw AI outputs into valuable solutions that address user needs.

Key Responsibilities:

Oversee AI evaluation processes and ensure adherence to quality standards. Collaborate with engineering to generate AI outputs for various applications. Train and guide evaluators to produce high-quality data. Analyze evaluation results to identify trends and insights. Prioritize data requests in conjunction with product teams.

Must-Have

Skills:

Proficiency in AI evaluation and quality management. Strong project management and operational skills. Excellent communication and cross-functional collaboration abilities.

Industry

Experience:

Experience in the financial services or tax technology domain is preferred.