More jobs:
Content Evaluator – Customer Support & LLM Benchmarking
Job in
534001, Eluru, Andhra Pradesh, India
Listed on 2026-09-09
Listing for:
Innodata Inc.
Full Time
position Listed on 2026-09-09
Job specializations:
-
Entertainment & Gaming
Customer Service Rep
Job Description & How to Apply Below
We are seeking detail-oriented evaluators to conduct human quality evaluations for an enterprise AI customer support product on Instagram, Whats App, and Messenger. You wil evaluate and benchmark AI model responses against complex evaluation rubrics using provided business knowledge bases.
Key Responsibilities
● Model Evaluation:
Review and score AI-generated customer interactions across Foundational, Experiential, and Operational dimensions (e.g., Action Fidelity, Faithfulness, Halucination, Compliance, Tone, and Handoff).
● Intent & Fact Verification:
Benchmark both informational (R1) and transactional (R2) customer queries against authoritative business sources (FAQs, product catalogs, SOPs) within the task UI.
● Quality Assurance:
Participate in dual-review processes and daily calibration audits to ensure inter-rater agreement and establish ground-truth performance targets.
● Performance Targets:
Deliver precise evaluations within a fast-paced daily turnaround cycle (~45-minute average handle time per job).
Required Qualifications & Skills
● Customer Service Background:
Prior experience in customer service, cal centers, retail, or handling customer communications via email, chat, or phone (highly prioritized).
● English Proficiency:
Exceptional written English skils with a strong command of tone, brand voice, grammar, and nuance.
● Analytical Precision:
Ability to strictly fo low multi-tier evaluation guidelines, complex logic trees, and technical rubrics without deviation.
● Tech Adaptability:
Comfort using dedicated web-based tools and labeling interfaces.
Work Environment & Schedule
● Duration:
Ful-Time, 2-Month Contract (~60 working days), with potential extension based on project needs.
● Shift Alignment:
Fixed shift during US night hours
Disclaimer :
Please be aware of recruitment scams involving individuals or organizations falsely claiming to represent employers. Innodata will never ask for any form of payment during any stage of the recruitment, onboarding or employment process. If you believe you've been targeted by a recruitment scam, please immediately report it to
To View & Apply for jobs on this site that accept applications from your location or country, tap the button below to make a Search.
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
Search for further Jobs Here:
×