Sr Data Engineering; Foundation Models - W2

We are building next-generation foundation models for machine-generated data — including time series, logs, and large-scale event streams. We’re looking for a strong data engineer to design and scale the data pipelines that power model training and production systems.

This role sits at the intersection of distributed systems, data infrastructure, and machine learning.

• Build and scale distributed data pipelines for large-scale time series and log data
• Design reliable, high-performance Spark/Python workflows for model training datasets
• Analyze and resolve performance bottlenecks (latency, memory, skew, throughput)
• Improve data quality, validation, and reproducibility for ML workloads
• Partner with ML engineers and researchers to accelerate foundation model development
• Measure and optimize application and transaction performance in production systems
• 5+ years of software engineering experience
• Strong proficiency in Python
• Hands‑on experience with Apache Spark (PySpark or Scala)
• Experience working with time series, logs, or high‑volume event data
• Strong debugging and performance optimization skills

• Experience supporting ML or large model training workflows
• Familiarity with sequence modeling or time series data systems
• Experience with streaming systems (Kafka, Spark Streaming)
• Experience with cloud‑native or Kubernetes‑based platforms

You are pragmatic, data‑driven, and comfortable operating in ambiguous, research‑heavy environments. You can debug distributed systems, reason about data correctness at scale, and build infrastructure that researchers trust.