Data Engineer

Overview

BT-151 – Data Engineer

Skill Level: Mid

Location:

Chantilly/Herndon

MUST HAVE AN ACTIVE TS OR TS/SCI CLEARANCE TO APPLY. Those without an active security clearance will not be considered.

As a Data Engineer on the Data Catalog and Governance team, you will be the technical engineer that brings our governance policies to life. You will be responsible for the hands-on development and maintenance of the systems that connect our data sources to our enterprise data catalog. Working under the guidance of senior technical and program leadership, you will build the data pipelines and integrations necessary to automate the collection of metadata and lineage.

This is an implementation-focused role for a skilled engineer who enjoys solving technical challenges and wants to apply their skills to build a foundation of trust and transparency in data.

• Develop, maintain, and enhance data pipelines that automatically harvest metadata from various data sources (e.g., databases, data lakes, APIs) and ingest it into the enterprise data catalog (e.g., Collibra).
• Implement custom integrations using the data catalog's APIs and SDKs to connect with platforms like Databricks, Snowflake, and S3.
• Work with senior engineers and analysts to implement technical data quality rules and monitoring solutions within data pipelines.
• Support the technical administration and operational health of the data catalog platform, troubleshooting ingestion issues and ensuring system stability.
• Write and maintain clear, concise documentation for the data pipelines and integrations you build.
• Collaborate with data source owners and other engineering teams to ensure successful metadata extraction and lineage tracing.
• Contribute to the scripting and automation of recurring data governance tasks.

• 4-7 years of hands-on experience in a data engineering role.
• Strong programming proficiency in Python and advanced SQL.
• Experience building and maintaining data pipelines and ETL/ELT processes.
• Hands-on experience working with REST APIs to integrate different systems.
• Experience with cloud data platforms (e.g., Databricks, Snowflake, AWS S3, Azure Data Lake).
• A foundational understanding of metadata management and data governance concepts.

• Hands-on experience developing integrations for an enterprise data catalog platform (e.g., Collibra, Alation).
• Experience with workflow orchestration tools (e.g., Airflow, Prefect).
• Familiarity with Infrastructure-as-Code (e.g., Terraform).
• Exposure to streaming data technologies (e.g., Kafka).