Lead Data Engineer

CommIT Enterprises

CommIT Enterprises, Inc. is seeking a highly skilled Lead Data Engineer to join a team in Charleston, SC, but this role can be REMOTE. The position involves designing, developing, and maintaining advanced data ingestion pipelines and models that support the United States Marine Corps (USMC) logistics situational awareness and decision‑making. The role requires expertise in ETL/ELT processes, CI/CD pipeline development, and data modeling to ensure reliable, scalable, and secure data solutions.

The work directly supports the USMC by enabling data‑driven situational awareness, ensuring leaders have timely, accurate, and actionable insights to inform operational decisions.

Established in 2001, CommIT is a Certified Veteran‑Owned Small Business (CVOSB) providing innovative technical engineering and data science services. Our enterprise systems support includes the Department of Defense’s GCSS‑MC, CAC2S, TBMCS‑MC, and the Department of Veteran’s Affairs’ telehealth communications. We offer acquisition management, systems engineering, Agile software development, cloud management, IT modernization, data analytics, cybersecurity, and training, including leading‑edge Dev Sec Ops , automated testing, and mobile application development.

• Lead the design, development, and maintenance of CI/CD pipelines that enable ingestion teams to seamlessly move ETL/ELT processes through Development, Testing, and Production environments.
• Build and optimize ETL/ELT workflows to ingest, validate, and transform data from diverse sources.
• Apply data modeling techniques to create Silver and Gold Tier data assets that enhance situational awareness and support decision‑making.
• Collaborate with cross‑functional teams to ensure data pipelines and models align with operational requirements and reporting needs.
• Drive best practices in data engineering, automation, and pipeline reliability to support mission‑critical analytics.

• Master’s degree with 6 years of experience (or Bachelors with 8 years of experience) in Computer Science, Software Engineering, Computer Engineering, Mathematics or relevant field. Degree may be substituted with additional relevant industry experience and/or industry accepted training and certification.
• 5+ years of experience in data engineering with a focus on ETL/ELT and CI/CD.
• Hands‑on experience with Databricks, Delta Lake, and Git Lab Enterprise.
• Strong proficiency in Python and SQL.
• Experience supporting logistics, defense, or mission‑critical environments is a plus.
• Proven experience with ETL and ELT processes for ingesting structured, semi‑structured, and unstructured data sources.
• Deep understanding of CI/CD practices, with the ability to design and implement pipelines using Databricks and Git Lab Enterprise for Data Ingestion teams.
• Strong background in data modeling, with the ability to build Silver and Gold Tier Tables and Views to support operational dashboards and reporting.

• ETL and ELT methods
• Databricks and Delta Tables
• Python and SQL programming
• CI/CD pipeline development and automation
• Git Lab Enterprise for version control and deployment

• Secret Clearance
• Security+ Certification

CommIT Enterprises, Inc. is an Equal Opportunity Employer. Employment decisions are made without regard to race, color, religion, national origin, gender, sexual orientation, gender identity, age, physical or mental disability, genetic factors, military/veteran status or other characteristics protected by law.