Psychometrician, AWS Training & Certification

Psychometrician, AWS Training & Certification

Job :  | Amazon Web Services, Inc.

The Amazon Web Services (AWS) Certification team is seeking a Psychometrician with experience working with criterion‑referenced assessment programs to support a large global AWS Certification and Credentialing program. You will work closely with a team of psychometricians, subject matter experts, certification exam program managers, publishing, delivery, security, and product management teams to support ongoing analyses of exam and credential data. To be successful in this position, you must be highly motivated, creative, detail oriented, and a self‑starter who can think big, execute, ensure high quality, yet stay focused on the details.

• Perform and support the main psychometric aspects of exam development and operations, including automated test assembly, item and test analyses, optimal item bank design, job task analysis, standard setting, quality assurance, and project planning.
• Conduct item analysis using psychometric methods, build optimal test forms and pools via optimization techniques, analyze and monitor item bank health, set pass standards via standard setting studies, and support Job Task Analysis (JTA) to define and refresh test blueprints.
• Develop and apply statistical and psychometric modeling to evaluate and ensure AWS certification exams’ validity, reliability, applicability, efficiency, and accuracy.
• Participate in research projects to improve operational processes and quality using advanced techniques such as machine learning (ML), statistical modeling, natural language processing (NLP), and generative artificial intelligence (GenAI).
• Develop automation code using R or Python for psychometric workflow pipeline and other tasks to enhance operational efficiencies.
• Present, interpret, and communicate analysis results to stakeholders through written and oral reports.
• Follow accreditation standards set by ISO/IEC: and the National Council for Certifying Agencies (NCCA) as they relate to valid psychometric practices.
• Engage with the professional community through conferences and publications.

• PhD in Statistics, Psychometrics, Educational Measurement, Quantitative Psychology, Data Science, Industrial‑Organizational (I/O) Psychology, or a related field.
• One year of research or work experience in the role offered, or experience as a Research Scientist, Research Assistant, Software Engineer, or a related occupation.
• At least one year of experience in the following areas:
  • Large‑scale education, licensure, or certification assessment programs.
  • Operational psychometric tasks on large‑scale assessment programs including item analysis, equating and scaling, item response theory, classical test theory, form and pool assembly, item bank health analysis, standard setting, and job task analysis.
  • At least one complex test design such as linear‑on‑the‑fly testing (LOFT) or computerized adaptive testing (CAT).
  • At least one of machine learning (ML) or natural language processing (NLP).
  • Programming skills in at least one script‑based programming language (R, Python).

• Experience in testing for global audiences.
• History of conference presentations and publications.
• Experience with data forensics methods.

Amazon offers comprehensive benefits including health insurance (medical, dental, vision, prescription, Basic Life & AD&D insurance, optional Supplemental life plans), Employee Assistance Program (EAP), Mental Health Support, Medical Advice Line, Flexible Spending Accounts, Adoption and Surrogacy Reimbursement coverage, 401(k) matching, paid time off, and parental leave. Learn more about our benefits at (Use the "Apply for this Job" box below)..

The base salary range for this position is listed below. Your Amazon package will include sign‑on payments and restricted stock units (RSUs). Final compensation will be determined based on factors including experience, qualifications, and location.

• USA, OR, Portland – $ – $ USD annually
• USA, TX, Austin – $ – $ USD annually
• USA, VA, Arlington – $ – $ USD annually
• USA, VA, Herndon – $ – $ USD annually
• USA, WA, Bellevue – $ – $ USD annually
• USA, WA, Seattle – $ – $ USD annually

Amazon is an equal opportunity employer and does not discriminate on the basis of protected veteran status, disability, or other legally protected status.

Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit  for more information. If the country/region you’re applying in isn’t listed, please contact your Recruiting Partner.