Data Engineer

Sorry we are not considering any candidates that require sponsorship or transfer of any kind as the client needs to be able to convert this person without any kind of transfer or sponsorship of any visa. Local candidates only.

Data Engineer to help build and scale its modern cloud data platform. This role will be responsible for designing and developing data pipelines, integrating internal and external data sources, and building scalable cloud solutions using Azure Databricks
, Amazon Web Services (AWS), and Python
.

The ideal candidate is passionate about cloud technologies and data engineering, enjoys building applications and automation tools, and has experience creating reliable, high-performance data solutions that support analytics and business intelligence. This position will work closely with Data Scientists, Business Analysts, and business stakeholders to transform raw data into trusted, actionable insights.

• Design, develop, and maintain scalable ETL/ELT pipelines using Azure Databricks and Py Spark .
• Build robust data integration solutions for structured and unstructured data sources.
• Design and optimize data models to support reporting, analytics, and machine learning.
• Monitor, troubleshoot, and optimize data pipelines for performance, reliability, and scalability.
• Implement data quality checks and governance best practices.
• Design and maintain cloud-based data storage solutions using Amazon S3
  .
• Configure secure AWS environments for data ingestion from internal and third-party data providers.
• Build automated processes for ingesting files into S3 and orchestrating downstream processing.
• Work with AWS services including:
• Amazon S3
• AWS Glue
• Lambda
• IAM
• Cloud Watch
• Support future cloud-native initiatives utilizing AWS container technologies such as ECS, EKS, or Kubernetes.

• Develop internal applications, automation tools, APIs, and utilities using Python
  .
• Build reusable services that streamline business processes and improve operational efficiency.
• Collaborate with cross-functional teams to deliver scalable, data-driven solutions.
• Partner with Sales, Marketing, Finance, Supply Chain, and Operations teams to understand business requirements.
• Support Data Scientists by preparing high-quality datasets for advanced analytics and AI initiatives.
• Work closely with IT and business stakeholders to implement enterprise data solutions.

• Bachelor's degree in Computer Science, Information Systems, Data Engineering, Software Engineering, or a related field.
• 3+ years of experience in Data Engineering or a similar role.
• Hands-on experience with Azure Databricks
  .
• Experience working within Amazon Web Services (AWS).
• Strong programming skills in Python
  .
• Advanced SQL development skills.
• Experience building scalable ETL/ELT pipelines.
• Experience with Git and source control.
• Strong understanding of relational databases and data warehousing concepts.

• Experience working with syndicated market data providers such as NielsenIQ
  , Circana (IRI), SPINS, or retailer POS data.
• Experience with:
• Py Spark
• Spark SQL
• Snowflake
• Amazon Athena
• Amazon Redshift
• Experience with Docker, Kubernetes, or containerized applications in AWS.
• Familiarity with CI/CD pipelines and Dev Ops practices.