Data Engineer

Data Engineer

Location:

Chantilly, VA

Clearance:
Active TS/SCI w/ Polygraph needed to apply

Develop new tools, code, and services to execute data engineering activities involving data of varying types and in varying conditions. Activities include the following tasks: movement of structured and unstructured data using approved methods; executing data ingestion activities for storing data in local or enterprise-level locations; developing code to format data that supports exploration; analyzing source data formats and working with Data Scientists and partners to determine the formats and transforms that best meet mission objectives;

developing code and tools to provide one-time and ongoing data extraction from various repositories, formatting and transformations into enterprise or standalone data models; creating new ETL and performing O&M and enhancements on existing ETL code using best practices and standards; and delivering documentation for each project including ETL mappings, code use guide, code location, and access instructions.

• Design and optimize Data Pipelines using tools such as Spark, Apache Iceberg, Trino, Open Search, EMR cloud services, NiFi and Kubernetes containers
• Ensure the pedigree and provenance of the data is maintained such that the access to data is protected
• Clean and preprocess data to enable access for advanced analytics
• Collaborate with enterprise working groups to advance the state of data standards
• Collaborate with the engineering team, data stewards, and mission partners to aid in getting actionable value out of the data holdings
• Collaborate with software engineers to update, configure, and maintain data services based on the requirements
• Ensure data quality by working with the testing and data quality team to enhance standardization of data conditioning pipelines
• Experience adapting to various types and formats of data, and working with development teams to integrate new data processing platforms

• 10+ years' experience with data lifecycle engineering
• Development and maintenance of extract, transform and load (ETL) tools and services
• Cloud and on-prem data storage and processing solutions
• Python, SQL, Spark and other data engineering programming
• COTS and open source data engineering tools such as Elastic Search and Ni Fi
• Processing data within the Agile Lifecycle

• Medical, Dental and Vision Plans
• Generous PTO Policy
• 401(k)
• HSA and FSA options
• Life and Disability Insurance
• Tuition Reimbursement and Training
• Perks at Work Discount Program
• Referral Program
• Leads Generation Program
• College America 529
• Fitness Reimbursement Program
• Travel Assistance
• Norton Lifelock Benefit Solutions
• Life Planning Financial & Legal Services