Skip to content
×

Sr Data Scientist

Listing for: Amazon Development Center U.S., Inc.
Full Time position Listed on 2026-11-15
Job specializations:
Salary/Wage Range or Industry Benchmark: 183000 USD Yearly USD 183000.00 YEAR

Job Description & How to Apply Below

This role focuses on leading the evaluation and benchmarking data development that supports Amazon Quick. You will design datasets, define metrics and evaluation frameworks, and use the resulting insights to guide product decisions and model improvements.

Responsibilities
  • Design and build evaluation and benchmark datasets.
  • Define quality metrics and evaluation frameworks, including human evaluation, model-based evaluation using LLM-as-a-Judge
    , and checks on the correlation between human and model evaluations.
  • Run human annotation programs end to end, including writing guidelines, measuring inter-annotator agreement, and managing data quality controls.
  • Analyze model and agent performance to identify gaps, root causes, and the most impactful improvement opportunities.
  • Collaborate with science, engineering, and product teams to translate evaluation results into launch decisions and roadmap priorities.
  • Present findings and recommendations to senior leadership.
Requirements
  • 5+ years of experience with data querying languages (for example,
    SQL
    ), scripting languages (for example,
    Python
    ), or statistical and mathematical software (for example, R,
    SAS
    , Matlab
    ).
  • 4+ years of data scientist experience.
  • Experience leading and mentoring on a data science team.
Preferred Qualifications
  • 2+ years of experience in data visualization using AWS Quick Sight
    , Tableau
    , R Shiny
    , or similar tools.
  • Experience managing data pipelines.
  • Experience evaluating LLMs
    , RAG systems, or agentic AI applications.
Technologies

SQL
, Python
, R,
SAS
, Matlab
, AWS Quick Sight
, Tableau
, R Shiny
, LLM-as-a-Judge
, RAG

Compensation and Location

Location: Santa Clara, CA (onsite).
Salary: USD 183, per year.

Benefits
  • Comprehensive benefits including health insurance (medical, dental, vision, prescription, Basic Life & AD&D insurance and option for Supplemental life plans), EAP, mental health support, medical advice line, flexible spending accounts, and adoption and surrogacy reimbursement coverage.
  • 401(k) matching.
  • Paid time off.
  • Parental leave.
  • Sign-on payments and restricted stock units (RSUs).
To View & Apply for jobs on this site that accept applications from your location or country, tap the button below to make a Search.
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
View / Apply for Jobs
Matching My Jurisdiction

Search for further Jobs Here:

SEARCH JOBS: 10+ Million Listings
(Try combinations for better Results! Or enter less keywords for broader Results)
Education Level
Experience Level (years)
Posted in last:
Salary