Senior Data Engineer

Direct message the job poster from Pinnacle Government Solutions

Pinnacle Government Solutions is a minority-owned small business that provides Software and Cloud Engineering Solutions for our Department of Defense and Federal Civilian agencies. We bring SME-level expertise and over two decades of industry knowledge and experience. We believe in getting it right for our customers and our country.

We are seeking a Data Engineer that will develop new tools, code, and services to execute data engineering activities involving data of varying types and in varying conditions. Activities include the following tasks:

• Movement of structured and unstructured data using approved methods.
• Execute data ingestion activities for storing data in a local or enterprise level location.
• Develop code to format data that supports exploration.
• Analyze source data formats and work with Data Scientists and partners to determine the formats and transforms that best meet mission objectives.
• Develop code and tools to provide one-time and on-going data extraction from various repositories, formatting and transformations into enterprise or standalone data models.
• Develop new ETL and perform O&M and enhancements on existing ETL code using best practices/standards.
• Develop and deliver documentation for each project including ETL mappings, code use guide, code location and access instructions.

• Design and optimize Data Pipelines using tools such as Spark, Apache Iceberg, Trino, Open Search, EMR cloud services, NiFi and Kubernetes containers.
• Ensure the pedigree and provenance of the data is maintained so that access to data is protected.
• Clean and preprocess data to enable access for advanced analytics.
• Collaborate with enterprise working groups to advance the state of data standards.
• Collaborate with the engineering team, data stewards, and mission partners to aid in getting actionable value out of the data holdings.
• Collaborate with software engineers to update, configure, and maintain data services based on the requirements.
• Ensure data quality by working with the testing and data quality team to enhance standardization of data conditioning pipelines.
• Adapt to various types and formats of data, and work with development teams to integrate new data processing platforms.

• Applicants must have active TS/SCI w/ polygraph and minimum of 10 years' experience.
• Data lifecycle engineering.
• Development and maintenance of ETL tools and services.
• Cloud and on-prem data storage and processing solutions.
• Python, SQL, Spark and other data engineering programming.
• COTS and open-source data engineering tools such as Elastic Search and NiFi.
• Processing data within the Agile Lifecycle.

• Experience using AI/LLMs to accelerate the processing and transformation of data.

Mid-Senior level

Full-time

IT Services and IT Consulting

Referrals increase your chances of interviewing at Pinnacle Government Solutions by 2x