Software Development Engineer

The Artificial General Intelligence (AGI) team is looking for a passionate, talented, and inventive Software Development Engineer to help advance the quality and capabilities of industry-leading large language models and Generative AI  this role, you will build scalable solutions for discovering, processing, curating, transforming, and delivering high-value data used across model evaluation, training, and post-training workflows. You will work with large-scale and heterogeneous datasets, including data produced by partner teams, long-context data, and other data sources that can help improve model development.

You will collaborate closely with scientists, ML engineers, evaluation teams, and other engineering teams to identify valuable data sources, understand downstream data requirements, and develop mechanisms that make data easier to discover, standardize, validate, transform, and consume r work will help expand the breadth, quality, and usability of data available for Generative AI development and establish scalable data infrastructure, quality standards, and workflows that enable downstream teams to evaluate and improve models more effectively.

This is an opportunity to solve challenging problems at the intersection of data infrastructure, evaluation data, and Generative AI, and to directly influence the data foundation supporting Amazon’s next generation of AI models and experiences.

Key job responsibilities

Quickly learn and apply emerging technologies, methodologies, and best practices in Generative AI, data processing, and large-scale model development.

Design, build, and maintain scalable platforms and infrastructure for discovering, scanning, processing, curating, and delivering data used to support LLM evaluation, training, and model improvement.

Develop systems and mechanisms to identify useful evaluation data produced by partner teams, understand its characteristics and usability, and make relevant data easier to discover, standardize, validate, and consume.

Expand the breadth and quality of data available for model development, including evaluation datasets, long-context data, training data, and other high-value data sources.

Build scalable capabilities to transform heterogeneous data from different sources and formats into standardized, high-quality datasets that can be reliably consumed by downstream evaluation and training workflows.

Partner closely with Applied Scientists, ML Engineers, evaluation teams, and other engineering teams to understand data needs, identify gaps in available data, and translate those needs into scalable data solutions.

Investigate design approaches, prototype new technologies, evaluate technical feasibility, and drive data infrastructure solutions from concept to production.

Build reliable, high-quality data discovery and processing pipelines that operate at massive scale while improving data quality, coverage, observability, efficiency, and developer productivity.

Influence technical architecture and establish engineering best practices for data discovery, curation, transformation, and delivery across Generative AI model development workflows.

A day in the life

As an SDE with the AGI team, you will build scalable systems that discover, process, curate, transform, and deliver high-value data for model evaluation, training, and post-training workflows.

You will work with large-scale, heterogeneous data sources and partner closely with scientists, ML engineers, and evaluation teams to understand downstream data needs and make useful data easier to identify, standardize, validate, and consume.

You will influence technical architecture, establish engineering best practices, and build reliable infrastructure that improves data quality, usability, and…