Gen AI--Icelandic & Luxembourgish

Multilingual AI Evaluation Specialist

Locales:
Icelandic & Luxembourgish

Raters required: 5 per locale (Native)

Remote role

Initial Commitment: 20 hours

Role Overview:
Multilingual AI Evaluation Specialist

We are seeking detail-oriented language experts to spearhead our Multilingual Benchmarking Initiative. In this role, you will be responsible for creating, refining, and executing evaluation benchmarks that ensure our AI and Large Language Models (LLMs) are not just linguistically accurate, but also culturally resonant and aligned with regional expectations.

Key Responsibilities

• Benchmark Creation:
  Design and develop comprehensive evaluation datasets tailored to your native language’s unique structure and contemporary usage.
• Cultural Localization:
  Assess and adapt content to ensure it aligns with regional cultural contexts, taboos, humor, and societal norms.
• Linguistic Quality Assurance:
  Evaluate AI-generated outputs for grammatical precision, natural phrasing, appropriate syntax, and accurate pronunciation/phonetics (where applicable).
• Dialect & Variation Analysis:
  Ensure the benchmarks account for regional dialects, slang, and colloquialisms to capture true local diversity.

Project Scope & Effort Required

• Initial Commitment: 20 hours per locale for the foundational benchmarking phase.
• Potential for Extension:
  This is an initial estimate. Depending on project velocity, evaluation complexity, and specific regional needs, the scope and hours may scale up significantly.
• Flexibility:
  Work can be completed remotely and asynchronously, provided strict project milestones and deadlines are met.

Required Qualifications

• Native Fluency:
  Absolute native-level proficiency (written and spoken) in the target language, paired with deep immersion in the corresponding regional culture.
• Linguistic Expertise: A flawless grasp of grammar, mechanics, idiomatic expressions, and regional linguistic variations.

Preferred Experience

• Prior experience in data annotation, linguistic evaluation, translation QA, or AI/LLM reinforcement learning (RLHF).
• A background in linguistics, literature, communication, or a related field.
• Familiarity with AI evaluation frameworks, prompt engineering, or prompt-response testing.