Skip to content
×

Adversarial Evaluator, AI Search Red Teaming Evaluator

Job in Boise, Ada County, Idaho, 83701, USA
Listing for: Truelancer AI
Part Time position Listed on 2026-11-12
Job specializations:
Salary/Wage Range or Industry Benchmark: 55000 - 91000 USD Yearly USD 55000.00 91000.00 YEAR

Job Description & How to Apply Below

Contract
· Remote
· 5-week pilot
· 15 openings

About the role

Truelancer AI is building a specialist red team for an AI search safety pilot with a global enterprise client. You’ll work as an individual contributor. You’ll own specific attack surfaces, design and run adversarial scenarios against an AI search and RAG system, and produce evidence that holds up under QA review.

We value complete evidence and defensible coverage more than prompt volume.

What you’ll do
  • Plan attacks: form risk hypotheses and design original, bounded adversarial scenarios.

  • Run attacks: execute multi-turn, adaptive prompt attacks, including context escalation, on authorised systems.

  • Evaluate outputs for these failure types:

    • grounding errors
    • source contradictions
    • stale information
    • unverifiable citations
    • trust-boundary violations, such as indirect prompt injection
  • Record evidence that can be reproduced:

    • exact prompt sequences
    • session and version context
    • timestamps
  • Submit finding cards to QA, and adjust your judgements as rubrics and guidelines are updated.

  • Classify each outcome as a candidate finding, a quality issue or inconclusive. Know when to stop and elevate an unsafe or uncertain case.

  • Stay within scope: keep to client-approved boundaries and privacy policies at all times.

  • Keep current on client taxonomies, system changes, policy revisions and new risk definitions.

  • Work with the team: the Red-Team Lead, Project Manager and QA analysts.

Who you are
  • Experience: 3 or more years in one or more of the following:

    • AI red teaming, LLM evaluation or GenAI testing and QA
    • Trust & Safety, content safety or content policy operations
    • Fraud, abuse or risk investigations
    • Responsible AI, AI governance or AI safety
  • Methods: hands-on experience with LLM and model evaluation, RLHF or human-feedback work, and prompt engineering.

  • Adversarial skill: working knowledge of prompt injection, jailbreak testing and adversarial techniques.

  • Mindset: sceptical and analytical. You can rule out false positives and check citations against sources.

  • Communication: excellent written and spoken US English, suited to structured reporting.

  • Detail: very high attention to detail and ownership of what you deliver.

  • Education: graduate or undergraduate degree.

How you’ll be measured
  • Defensible coverage of your assigned surfaces, and how reliably your findings reproduce
  • QA calibration and adjudication scores, with a target of at least 85% agreement
  • Compliance with client boundary and privacy policies
  • Quality of your finding cards and how well you take on calibration updates
Guidelines

Truelancer AI verifies each project before posting.

To View & Apply for jobs on this site that accept applications from your location or country, tap the button below to make a Search.
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
View / Apply for Jobs
Matching My Jurisdiction

Search for further Jobs Here:

SEARCH JOBS: 10+ Million Listings
(Try combinations for better Results! Or enter less keywords for broader Results)
Education Level
Experience Level (years)
Posted in last:
Salary