Senior Data Engineer

Overview

The Data Engineer will be writing code that moves data through a pipeline, fixing data pipeline issues, optimizing data systems, and collaborating with stakeholders, day to day.

The Capital Software Project is building a data fabric system. The data fabric system includes a data pipeline that will be moving a complex mix of data types and formats to a data lake and a series of databases, allowing mission engineering analysts to conduct data analysis from authoritative sources of truth.

• Write or update ETL/ELT code that ingests data from APIs, databases, or files.
• Implement transformations to clean, standardize, or enrich data.
• Optimize pipeline performance and improve reliability.
• Hands-on experience with AI and ML pipelines, MLOps workflows, and data processing methodologies.
• Implement methods for data traceability and lineage.
• Monitor automated alerts and logs for failed jobs or data anomalies.
• Troubleshoot issues such as schema changes, missing data, or performance bottlenecks.
• Validate data accuracy and consistency across systems. Leverage your expertise in semantic modeling.
• Meet with project teams to understand new data requirements.
• Work with backend engineers or Dev Sec Ops  to align pipeline changes with system updates.
• Support stakeholders by providing data extracts or explanations of data structure.
• Plan new data models or schemas for analytics or application use case.
• Incorporate metadata strategies tailored for the use case.
• Help shape the data warehouse/lake structure.
• Evaluate new tools or cloud services to improve scalability and maintainability.
• Build reusable application containers in Python, SQL, or other languages used in the stack.
• Document pipeline logic, data contracts, and system diagrams.
• Participate in code reviews to ensure quality and best practices.
• Work location:
  
  Due to the security classification for this project, work will be performed in-office, 5-days a week.

• 7+ years of relative working experience as a Senior Data Engineer or in a related field.
• Strong experience with ETL/ELT development and building data pipelines.
• Proficiency with at least one major programming language (e.g., Python, Java, Scala).
• Experience working with SQL and relational databases (e.g., Postgre
  
  SQL, SQL Server, MySQL).
• Exposure and familiarity with data services on modern cloud platforms (AWS, Azure, GCP, etc.).
• Familiarity with modern data warehousing technologies (e.g., ADLS, Redshift, Big Query, Synapse).

• Bachelor’s degree Computer Science, Software Engineering, or a related field.

• Must have a Secret DoD Security Clearance.

#MTSI