AI Agent Evaluation Analyst; Freelance

At Mindrift, innovation meets opportunity. We use the collective human intelligence to ethically shape the future of AI.

We are seeking QAs for autonomous AI agents on a new project focused on validating and improving complex task structures, policy logic, and agent evaluation frameworks. The role is flexible, project‑based, and well suited for remote, part‑time contributors.

• Review evaluation tasks and scenarios for logic, completeness, and realism
• Identify inconsistencies, missing assumptions, or unclear decision points
• Help define clear expected behaviours (gold standards) for AI agents
• Annotate cause‑effect relationships, reasoning paths, and plausible alternatives
• Think through complex systems and policies as a human would to ensure proper testing of agents
• Work closely with QA, writers, or developers to suggest refinements or edge‑case coverage

• Excellent analytical thinking: reason about complex systems, scenarios, and logical implications
• Strong attention to detail: spot contradictions, ambiguities, and vague requirements
• Familiarity with structured data formats: read (not necessarily write) JSON/YAML
• Ability to assess scenarios holistically: identify missing, unrealistic, or breakable elements
• Good communication and clear writing (in English) to document findings

• Experience with policy evaluation, logic puzzles, case studies, or structured scenario design
• Background in consulting, academia, olympiads (logic/math/informatics), or research
• Exposure to LLMs, prompt engineering, or AI‑generated content
• Familiarity with QA or test‑case thinking (edge cases, failure modes, "what could go wrong")
• Some understanding of how scoring or evaluation works in agent testing (precision, coverage, etc.)

• Get paid for your expertise, with rates up to $80/hour depending on your skills, experience, and project needs
• Take part in a flexible, remote, freelance project that fits around your primary professional or academic commitments
• Participate in an advanced AI project and gain valuable experience to enhance your portfolio
• Influence how future AI models understand and communicate in your field of expertise

Location: Remote (flexible scheduling)

Seniority Level: Internship

Employment Type: Part‑time