Data Engineer; Poly

Data Engineer

Insight Global is seeking an experienced Data Engineer to support us and a classified intelligence customer in developing tailored solutions to tackle unique global collection problems. The work may be performed independently or within a team environment depending on the specific problem statement. Work will include developing Amazon Web Services (AWS)-based resources – we need skills spanning many compute, storage, and networking services.

Duties:

• Work closely with the Customer's technical lead to architect, deploy, and maintain multiple, fast-turnaround capabilities used to perform various highly-visible and high-priority collection efforts.

• Strategically apply AI/ML to extract, format, and expose in indexed search tools relevant content such as raw text, multimedia (audio, image, video, document), tabular (CSV, Parquet, Avro) or nested (JSON, JSONL, XML), and other structured /unstructured data types. Data is expected to be of varying formats, schemas, and structures.

• Provide Data Engineering support to include cleaning, modeling, and formatting data of unknown formats.

• Move data between different cloud storage environments for critical requests.

• Coordinate with multiple entities, including mission partners, to ensure capabilities and deliverables meet defined requirements and tradecraft needs.

• Create and maintain collection capabilities and deliverables within the Customer's Amazon Web Services environment utilizing Customer approved AWS services.

• Validate collected data to ensure it meets data format requirements.

• Maintain all source code in Customer's Git Hub repository.

• Document all source code, including how to execute the code.

• Perform operations and maintenance on the collection capabilities and deliverables to adapt to changes in collection target, technologies, data formats, and naming conventions.

Hourly: 84-91/hr Salary: ,000