Data Engineer

CTEC is a leading technology firm that provides modernization, digital transformation, and application development services to the U.S. Federal Government. Headquartered in McLean, VA, CTEC has over 300 team members working on mission-critical systems and projects for agencies such as the Department of Homeland Security, Internal Revenue Service, and the Office of Personnel Management. The work we do effects millions of U.S. citizens daily as they interact with the systems we build.

Our best-in-class commercial solutions, modified for our customers’ bespoke mission requirements, are enabling this future every day.

The Company has experienced rapid growth over the past 3 years and recently received a strategic investment from Main Street Capital Corporation (NYSE: MAIN). In addition to our recent growth in Federal Civilian agencies, we are seeking to expand our capabilities in cloud development and footprint in national-security focused agencies within the Department of Defense and U.S. Intelligence Community.

We are seeking to hire a Data Engineer to our team!

Client: CTEC develops and delivers innovative customer-centric technologies and solutions that support the Office of Personnel Management’s (OPM) Health and Insurance business unit and Office of the Chief Information Officer (OCIO).

• Design, develop, and maintain scalable ETL pipelines and data workflows to ingest, transform, and integrate data from legacy systems and external sources into modern cloud-based data platforms.
• Build, optimize, and maintain data processing solutions using Azure Databricks and lakehouse architectures to support analytical, operational, and reporting use cases.
• Support phased data migration from legacy databases and ETL tools to Azure Databricks environments, including transformation documentation and data mapping.
• Implement layered lakehouse data architectures (e.g., bronze, silver, gold layers) in Databricks to support data quality, performance, and downstream reporting needs.
• Develop data processing notebooks, workflows, and distributed data transformations using Python and PySpark within Databricks environments.
• Develop data validation, reconciliation, and testing processes to ensure data accuracy, completeness, and consistency across data domains.
• Integrate Databricks data platforms with analytics and reporting tools to enable business intelligence and operational dashboards.
• Support data governance initiatives including metadata management, data catalog integration, encryption, access controls, and compliance with federal data protection requirements.
• Maintain source control and CI/CD pipelines for Databricks and data engineering workflows, supporting automated promotion across environments.
• Work closely with data architects, solution architects, business analysts, and reporting teams to implement approved data solutions.
• Provide ongoing support for Databricks workflows, resolve pipeline failures, and troubleshoot complex data processing issues.
• Provide guidance to junior data engineers and contribute to documentation and team enablement.
• Work independently with minimal supervision.

• 7+ years of experience in data engineering, ETL development, or large-scale data integration environments.
• Strong experience designing and developing ETL pipelines and data transformations in Azure Databricks environments.
• Proficiency in SQL and Python, with hands‑on experience using PySpark for distributed data processing.
• Experience working with cloud-based data platforms, data lakes, and lakehouse environments, preferably on Microsoft Azure.
• Experience implementing layered lakehouse data architectures (bronze, silver, gold) for enterprise analytics.
• Familiarity with Spark-based big data processing frameworks.
• Experience supporting data migration from legacy databases and ETL tools to Databricks-based platforms.
• Experience integrating Databricks platforms with business intelligence and reporting tools such as Power BI.
• Familiarity with data governance, metadata management, and data security best practices.
• Experience with source control and CI/CD pipelines for data engineering and Databricks workflows.
• Worki…