Data Engineer – Data Integration C2C : Atlanta, GA

Data Engineer – Data Integration

Position:
Data Engineer – Data Integration

Location:

Atlanta, GA (onsite)

Type:
Long-term

Job description:

A Data Engineer with expertise in Data Integration is responsible for designing and building solutions to transfer data from operational and external environments to the business intelligence environment. They utilize tools such as Informatica, Ab Initio software, and Data Stage (formerly Ascential) – Data Integration Suite. This role involves creating and implementing Extract, Transform, and Load (ETL) processes, ensuring the seamless flow of data throughout the business intelligence solution’s lifecycle.

• 7+ years of experience in Data Engineering / Data Migration / ETL with strong hands‑on experience in AWS DMS for full‑load and CDC‑based data movement.
• Strong experience with Python for automation, data processing, validation, reconciliation, and operational scripting.
• Hands‑on experience with Terraform for provisioning and managing AWS cloud infrastructure.
• Strong working experience with Snowflake including data loading, staging, SQL development, performance tuning, and data validation.
• Experience fetching data from various enterprise sources including databases, S3 buckets, and SFTP, with exposure to Kerberos authentication for secured database connectivity.

• Experience with full‑stack development using Python‑based frameworks.
• Experience with AWS services such as Lambda, Glue, Step Functions, Event Bridge, IAM, Secrets Manager, Cloud Watch, and VPC networking.
• Experience with large‑scale data migration, cloud modernization, or enterprise data warehouse implementation projects.
• Exposure to CI/CD tools such as Git Hub Actions, Git Lab CI, or AWS Code Pipeline.

• Fetch and process data from various sources including relational databases, Amazon S3, and SFTP servers.
• Work with Kerberos authentication to securely connect to enterprise databases and authenticated data sources.
• Build and manage AWS infrastructure using Terraform.
• Configure AWS services such as DMS, S3, IAM, VPC, Secrets Manager, Cloud Watch, and related cloud components.
• Load, validate, and optimize data in Snowflake using stages, file formats, warehouses, schemas, and SQL‑based transformations.
• Troubleshoot DMS replication failures, connectivity issues, authentication errors, latency, and performance bottlenecks.
• Perform data validation and reconciliation between source systems and Snowflake targets.
• Collaborate with application, database, infrastructure, security, and data engineering teams to deliver reliable migration solutions.
• Create technical documentation, data flow diagrams, deployment notes, runbooks, and support procedures.
• Support internal tools or dashboards using full‑stack development skills where required.

We need an experienced Cloud Data Engineer / Data Migration Engineer to support enterprise data movement and migration initiatives using AWS DMS, Python, Terraform, AWS, and Snowflake. The candidate will design, build, automate, and support secure data ingestion pipelines from multiple sources including enterprise databases, S3 buckets, and SFTP locations into Snowflake. The role requires exposure to Kerberos authentication for securely connecting to enterprise databases and protected data sources.

Design, configure, and support AWS DMS replication tasks for full-load and CDC‑based data migration.

Develop Python scripts for automation, data validation, reconciliation, file handling, and operational support.