Data Engineer, CPU & Storage

Data Engineer

OpenAI's Industrial Compute organization is building the world's most advanced AI infrastructure ecosystem. Through a combination of strategic partnerships and self-built campuses, we are scaling the compute, storage, and networking platforms that power frontier AI training and inference.

The Scaling Analytics team builds the data and software systems that help Industrial Compute understand, plan, and operate infrastructure at global scale. We work across capacity, hardware, storage, infrastructure software, and operational systems to connect fragmented sources of infrastructure data and make that information reliable and usable for engineering and planning.

As OpenAI's infrastructure footprint grows, CPU and storage data increasingly spans internal platforms, vendor systems, APIs, databases, object storage, capacity management systems, and operational tooling. Building reliable connections across these environments is critical to understanding available capacity, utilization, fleet state, and infrastructure growth.

We are seeking a Data Engineer to build the data systems and integrations that connect OpenAI's CPU, storage, and supporting infrastructure platforms.

This role sits at the intersection of data engineering and backend software engineering. Rather than focusing primarily on traditional analytical pipelines, you will build the software and integrations required to collect, normalize, and make infrastructure data available across a heterogeneous set of systems.

CPU and storage data may originate from internal infrastructure platforms, vendor APIs, databases, object storage, capacity systems, and operational services. You will determine how to reliably connect these systems and where those integrations should live—whether within an existing infrastructure service, an orchestration framework, a scheduled workload, or a purpose-built application.

You will work closely with Infrastructure Engineering, Capacity Engineering, Storage, Hardware Operations, and Infrastructure Software to create a reliable data foundation for understanding CPU and storage capacity, utilization, inventory, and operational state.

Success in this role requires strong software engineering fundamentals, comfort working across unfamiliar systems, and the ability to design pragmatic solutions for moving and reconciling data across infrastructure environments.

Responsibilities:

• Build and maintain software integrations that collect CPU, storage, capacity, inventory, and operational data from internal systems and external vendors.
• Connect heterogeneous data sources including APIs, relational databases, object storage, infrastructure services, capacity management platforms, and vendor systems.
• Design reliable ingestion mechanisms for both structured and semi-structured infrastructure data, including batch, scheduled, and API-driven workflows.
• Determine the appropriate architecture and execution environment for new data integrations, including existing backend services, capacity systems, orchestration frameworks, scheduled workloads, or purpose-built services.
• Build backend services, jobs, and tooling that normalize infrastructure data and make it available through databases, offline tables, and analytical datasets.
• Develop reliable interfaces between operational systems and downstream analytics, planning, and engineering workflows.
• Partner with infrastructure and software engineering teams to understand source systems, APIs, data ownership, schemas, and operational constraints.
• Design data models that reconcile information across multiple systems and establish consistent representations of CPU capacity, storage capacity, inventory, utilization, and infrastructure state.
• Build mechanisms for detecting missing, stale, inconsistent, or incorrect infrastructure data and resolving discrepancies between source systems.
• Improve the reliability, observability, and maintainability of data integrations through testing, monitoring, logging, and automated validation.
• Support infrastructure planning and operational analysis by ensuring critical CPU and storage datasets are accurate, timely, and accessible.
• Develop reusable patterns for…