Sr. Data Engineer - Services Special Projects

Sr. Data Engineer - Services Special Project

At Apple, great ideas have a way of becoming phenomenal products, services, and customer experiences very quickly. Our team is building a massive, real-time platform that transforms continuous streams of multimodal data (including structured, image, and log data) into an intelligent, searchable foundation. By enriching this data with language and embedding models, we power critical experiences for billions of Apple customers across multiple downstream applications.

We are seeking an experienced Data Engineer with deep expertise in ETL/ELT, data architecture, and applied ML pipelines to design, build, and operate this infrastructure. As a key member of the team, you will be responsible for creating the massively scalable pipelines that turn raw data into a trusted foundation, driving critical decision-making and operations across the entire system.

• Build and implement batch and streaming ETL/ELT pipelines that ingest, process, and model data from diverse sources, including unstructured media and real-time event streams, ensuring high reliability, performance, and scalability.
• Develop and maintain Kafka-based ingestion and processing pipelines, ensuring reliable data delivery across services and into the data lake.
• Build robust logical and physical data models with a focus on dimensional modeling, versioning, and storage patterns (e.g., Parquet, ORC) optimized for ingest, reporting, and operational use cases.
• Define and enforce data quality checks, SLAs, and observability standards to ensure data is accurate, timely, versioned, and trusted by stakeholders.
• Integrate and enrich raw signals with metadata and attribution to power downstream use cases such as analytics, billing, planning, and optimization.
• Implement standard methodologies for data lineage, metadata management, schema governance, versioning, and security in alignment with Apple's standards for data protection and privacy.
• Deliver solutions that include logging, anomaly detection, data validation, cleaning, and transformation, with strong emphasis on monitoring, debuggability, and continuous improvement.
• Work closely with ML engineers, data scientists, platform teams, and leadership to translate requirements into scalable, reliable data solutions.
• Help advance the team's data stack, including tooling, frameworks, and standards for development, testing, deployment, and operations.

• Masters Degree
• 10+ years of experience in data engineering, including building and maintaining large-scale ETL/ELT data pipelines
• Proficiency in data modeling, especially dimensional modeling, and designing schemas optimized for analytics and reporting
• Experience with leveraging databases including SQL/No
  
  SQL Databases (including Postgres / Cassandra / Redis)
• Strong experience with distributed data processing frameworks including Apache Spark
• Strong experience with Parallel processing frameworks:
  Big Table/Hadoop
• Strong software engineering fundamentals and proven experience with Scala, Java
• Hands-on experience with Apache Kafka, Iceberg, and Flink.
• Experience with workflow orchestration tools including Apache Airflow and Beam
• Experience with AWS: e.g., S3, EMR, Lambda, Glue, Redshift, Big Query, Kinesis, or similar services
• Experience with Analytics frameworks including Trino (Presto, Big Query, Snowflake)
• Hands-on experience with big data lake architectures
• Experience with containerization and orchestration (Docker, Kubernetes/EKS) and CI/CD tooling including Jenkins
• Experience in Python and Py Spark
• Familiarity with graph databases such as Tiger Graph
• Experience building pipelines that process multimodal data (structured and image) and integrate ML model inference - including LLMs and embedding models - for data enrichment and transformation
• Hands-on experience deploying, serving, and optimizing LLMs or ML models directly in the production, inference runtimes/compilers (ONNX Runtime, TensorRT/TensorRT-LLM), and serving frameworks (Triton, vLLM, Torch Serve or similar).
• Experience tuning batching, KV-cache, and GPU utilization for low-latency, high-throughput real-time inference in a data pipeline
• Kn…