Senior Data Engineer

Job Summary

We are seeking a Senior Data Engineer to architect and develop the core data infrastructure that will underpin our AI‑driven transformation. In this pivotal role, you will move beyond traditional ETL processes to design and deploy high‑security Data Lakehouse environments, establishing a single, authoritative source of truth for our AI systems. You will lead the technical implementation of ‘Workstream 1:
Data & Platform Foundations,’ a critical initiative for our flagship projects. Your responsibilities will include integrating complex enterprise systems such as SAP S/4

HANA and Ariba, as well as processing unstructured data like technical drawings and regulatory documents. By collaborating across teams, you will build high‑performance pipelines and systems that power Intelligent Supply Chain forecasting and generative AI tools. Operating within a structured ‘Sprint Zero’ framework, you will ensure robust data lineage, security, and compliance with defense‑grade standards. This role demands expertise in both structured and unstructured data engineering, with a focus on creating scalable, secure, and high‑performance data architectures.

Your work will directly enable AI agents to optimize procurement processes and support engineers in developing next‑generation systems, making you instrumental in transforming raw data into strategic advantage for national defense capabilities.

• Design and deploy defense‑grade Data Lakehouse architectures to serve as the Single Source of Truth for AI‑driven Supply Chain Intelligence, ensuring high‑performance pipelines and systems for Intelligent Supply Chain forecasting and generative AI tools.
• Lead the technical execution of Workstream 1:
  Data & Platform Foundations, mapping rigid enterprise systems (e.g., SAP S/4
  
  HANA, Ariba) and collaborating with cross‑functional teams to integrate and process complex unstructured data (technical drawings, regulatory text) for AI applications.
• Architect and deploy ingestion pipelines to extract high‑volume transactional data from ERP systems like SAP S/4
  
  HANA, Ariba, and PLM, ensuring near real‑time availability for forecasting models and AI agents.
• Build connectors for external market intelligence feeds (e.g., S&P Global, Orbis, Eco Vadis) to enrich internal procurement data with macroeconomic and geopolitical signals for enhanced decision‑making.
• Design and implement a standardized procurement data model and taxonomy across multiple entities, harmonizing fragmented datasets into a cohesive analytics layer.
• Engineer pipelines to ingest, process, and transform unstructured technical data (PDF tender documents, CAD metadata, historical CONOPS) into vector‑ready formats for Retrieval‑Augmented Generation (RAG) applications.
• Manage and optimize Vector Databases (e.g., Weaviate) to store embeddings of archival proposals and engineering snippets, ensuring high‑speed retrieval for AI drafting assistants and generative tools.
• Establish data lineage and traceability protocols to link requirements to physical components, supporting Model‑Based Systems Engineering (MBSE) and the Digital Thread implementation.
• Implement Role‑Based Access Control (RBAC), audit logging, and data redaction policies to ensure compliance with export controls and strict on‑premise security requirements.
• Deploy automated data quality frameworks to validate Bill of Materials (BOM) completeness and cost data accuracy before ingestion into AI models.
• Optimize data pipelines for on‑premise GPU clusters and air‑gapped environments, ensuring efficiency and performance within existing infrastructure constraints.
• Operate within a structured Sprint Zero environment to ensure data lineage, security, and governance meet defense‑grade standards.

• 5+ years of experience in Data Engineering, with at least 2 years focused on building pipelines for Machine Learning or Generative AI applications in an enterprise setting.
• Expert proficiency in Python, SQL, and modern data engineering frameworks including Apache Spark, Kafka, and Airflow.
• Strong experience extracting data from complex ERP environments, specifically SAP S/4
  
  HANA and…