Senior Software Engineer

AWS, Snowflake, data ingestion, data transformation, SQL, MLOps and Python

We are seeking a Senior Data Engineer to design, build and operate scalable data pipelines and analytical solutions using AWS and Snowflake. This role will integrate data from enterprise applications, APIs, databases and files, transforming it into reliable datasets for analytics, reporting and AI/ML use cases. The ideal candidate combines strong engineering skills with experience delivering secure, reliable and cost-effective data platforms in production.

• Design and maintain batch and incremental data ingestion pipelines from APIs, relational databases, SaaS applications and file sources.
• Build AWS ingestion solutions using services such as Amazon S3, AWS Glue, Lambda, IAM and Step Functions.
• Develop Snowflake data models and transformation pipelines using SQL, Python, dbt or equivalent tools.
• Implement Bronze, Silver and Gold layers within a medallion architecture, with clear data contracts and lineage.
• Process structured and semi-structured data, including JSON, CSV and Parquet.
• Implement change data capture, incremental loading, deduplication, schema evolution and historical data management.
• Integrate datasets across systems using consistent business keys, reference mappings and transformation rules.
• Build data quality checks, reconciliation controls, monitoring, alerting and recovery procedures.
• Optimize Snowflake queries, warehouse usage and storage to improve performance and control costs.
• Implement access controls, encryption, secrets management and data protection requirements.
• Establish automated testing, CI/CD and repeatable deployment practices for data pipelines.
• Troubleshoot production issues, perform root cause analysis and improve pipeline reliability.
• Collaborate with analysts, data scientists, architects and business stakeholders to translate requirements into technical solutions.
• Maintain technical documentation and provide engineering guidance through design and code reviews.

• 8+ years of professional experience in data engineering, data warehousing, ETL/ELT development or related roles.
• Strong hands‑on experience with AWS and Snowflake in production environments.
• Advanced SQL skills, including complex transformations, analytical functions and query optimization.
• Strong Python skills for API integrations, data processing, automation and error handling.
• Experience building ingestion pipelines with pagination, authentication, rate limiting, retries and checkpointing.
• Experience with Snowflake capabilities such as stages, storage integrations, COPY INTO, Snowpipe, Streams and Tasks.
• Experience with transformation frameworks such as dbt or equivalent SQL‑based tooling.
• Experience with orchestration tools such as Apache Airflow, AWS Step Functions or equivalent platforms.
• Strong understanding of dimensional modeling, slowly changing dimensions and incremental processing.
• Experience implementing data quality, observability, lineage and operational support practices.
• Proficiency with Git, code reviews and CI/CD.
• Strong problem‑solving, communication and stakeholder collaboration skills.

• Experience preparing and managing datasets for AI/ML, including feature engineering and training and inference pipelines.
• Familiarity with MLOps, including experiment tracking, model versioning, deployment, monitoring and retraining workflows.
• Experience with tools such as Amazon Sage Maker, MLflow, Snowpark or equivalent platforms.
• Familiarity with model drift, data drift and reproducibility.
• Experience supporting generative AI applications,…