Software Engineer, Data Engineering

Summary of Work:

The Sponsor requires web design services. The work will be performed independently as part of a larger team effort in a collaborative environment.

Required Skills:

• Demonstrated experience working in a
• Unix operating system to include usage of the Unix terminal.
• Demonstrated experience developing and maintaining data transformation scripts in Python.
• Demonstrated experience with writing code in at least one of the following programming languages:
  Java, Go, C#, or Scala.
• Demonstrated experience ingesting, transforming, modeling, and storing data in cloud environments (e.g., AWS, GCP, or Azure).
• Demonstrated experience with modeling structured, semi-structured, and unstructured data.
• Demonstrated experience working with SQL and No
  
  SQL databases to store and retrieve data (ex. Postgres SQL, MySQL, Elastic Search, MongoDB).
• Demonstrated experience with strong problem-solving skills and the ability to work in a fast-paced environment.

Desired

Skills:

• Demonstrated experience in the following concepts:
  Database, Data Warehouse, Datalake, and Lakehouse.
• Demonstrated experience with modern data flow management and orchestration tools. For example, Flyte, Apache Airflow, Dagster, or Apache NiFi.
• Demonstrated experience utilizing containerization technologies such as Docker, Podman, Kubernetes, or similar.
• Demonstrated experience ingesting, transforming, modeling, and storing data in the Sponsor’s cloud environments (e.g., AWS, GCP, or Azure).
• Demonstrated experience with AI/ML methodologies and techniques.
• Coordinate with multiple entities, including mission partners, to ensure tools meet defined requirements.
• Apply an agile approach to software development consistent with the Sponsor’s project management and software development frameworks with a focus on demonstrating and delivering releasable software every iteration.
• Ensure all security vulnerabilities are addressed as appropriate by severity and maintain security accreditation, including maintaining plan of action for vulnerability remediation.
• Design data pipelines using data ingestion patterns and business requirements.
• Build and deploy data pipelines in a cloud environment.
• Document data flows and data pipeline architectures.
• Perform end-to-end quality assurance of data.
• Troubleshoot and correct issues with existing data pipelines.
• Apply Dev Ops principles and philosophies to continuously delivery high value enhancements to software in a service-based architecture.
• Identify and document areas for improvement in workflows or systems for Sponsor’s consideration.
• Maintain all source code in Sponsor-wide, remote Git repository.