Data Engineer

About the Role

We're looking for a highly skilled and experienced Senior Data Engineer to join our team. You'll be instrumental in designing, building, and maintaining our next-generation Data Observability platform and its infrastructure. This role requires a deep understanding of distributed systems and hands‑on expertise with cutting‑edge big data technologies in a cloud‑native environment. You will be responsible for creating robust, scalable, and fault‑tolerant data pipelines that process massive datasets for both real‑time and batch‑based analytics.

Your work will directly enable our data scientists, analysts, and business teams to derive critical insights and make data‑driven decisions.

Location & In‑Office Requirements
You must be local to the Atlanta, GA area and adhere to our Return To Office (RTO) / weekly onsite requirements (Tuesday, Wednesday, and Thursday).

Immigration sponsorship
This position does not offer immigration sponsorship (current or future) including F‑1 STEM OPT extension support.

• Apply the knowledge of data characteristics and data supply pattern, develop rules and tracking process to support data quality model.
• Prepare data for analytical use by building data pipelines to gather data from multiple sources and systems.
• Integrate, consolidate, cleanse and structure data for use by our clients in our solutions.
• Perform design, creation, and interpretation of large and highly complex datasets.
• Stay up-to-date with the latest trends and advancements in GCP and related technologies, actively proposing and evaluating new solutions.
• Understand best practices for data management, maintenance, reporting and security and use that knowledge to implement improvements in our solutions.
• Implement security best practices in pipelines and infrastructure.
• Develop and implement data quality checks and troubleshoot data anomalies.
• Provide guidance and mentorship to junior data engineers.
• Review dataset implementations performed by junior data engineers.

• BS degree in a STEM major or equivalent discipline;
  Master’s Degree strongly preferred.
• 5+ years of experience as a data engineer or related role.
• Cloud certification strongly preferred.
• Advanced skills using programming languages such as Python or SQL and intermediate level experience with scripting languages.
• Intermediate level understanding and experience with Google Cloud Platform and overall cloud computing concepts, as well as basic knowledge of other cloud environments.
• Experience building and maintaining moderately‑complex data pipelines, troubleshooting issues, transforming and entering data into a pipeline for future projects.
• Experience designing and implementing moderately complex data models and enabling optimization to improve performance.
• Demonstrates advanced Git usage and CI/CD integration skills.

• 5+ years of hands‑on experience in data engineering or a related field.
• Expert‑level proficiency in SQL, Python or relevant programming languages.
• Demonstrated experience designing and implementing distributed data processing solutions at scale.
• In‑depth, hands‑on experience with Big Data tools such as Apache Spark or Apache Beam.
• Proven experience with Google Cloud Platform (GCP) services for data engineering, including Dataflow, Dataproc, Big Query, and Pub/Sub.
• Experience with workflow orchestration tools like Apache Airflow (or GCP’s Cloud Composer).
• Solid understanding of data modeling, data warehousing, and ETL/ELT best practices.
• Background in large data analysis and building AI/ML models would be a huge plus.
• Excellent problem‑solving skills and the ability to troubleshoot complex distributed systems.
• Effective communication and collaboration skills, with the ability to articulate technical concepts to both technical and non‑technical stakeholders.

Seniority level:
Mid‑Senior level

Employment type:

Full‑time
Job function:
Information Technology