Data Engineer Lead
-
Software Development
Job Description & How to Apply Below
Build production-ready data platforms at AIG, partnering closely with product and stakeholders while keeping engineering standards high. This hands-on Data Engineer Lead role combines strong pipeline delivery with leadership through reviews, mentorship, and documentation. You will also leverage GenAI tooling, including Claude Code and Snowflake Cortex
, as part of day-to-day development to improve code quality and productivity.
Based in Jersey City, NJ (onsite), this role includes a $125,000 to $135,000 base salary range and eligibility for a bonus under the terms of the applicable incentive plan.
What you’ll do- Design, build, and operate production-grade PySpark and Python pipelines on AWS EMR
, Glue
, and S3
, with integrated data quality checks and observability
. - Own end-to-end delivery of data products, from requirements through production readiness, working in an Agile/Scrum workflow that includes sprint planning
, CI/CD
, release management
, and production readiness. - Turn business and product requirements into concrete technical designs, including pipeline structure,
data models
, and defined SLAs
. - Apply architecture and design patterns with clear rationale, evaluating trade-offs and aligning decisions with peers and stakeholders.
- Set and enforce team engineering standards, including coding conventions
, data quality frameworks
, reusable pipeline patterns, and observability hooks. - Drive technical quality through thorough code reviews and pairing on difficult problems.
- Use GenAI tooling across the development lifecycle, including Claude Code CLI and Snowflake Cortex
, to improve solution quality and delivery speed. - Collaborate with product owners to refine requirements, surface trade-offs, and push back constructively when scope is not technically workable.
- Communicate design decisions and delivery updates to both technical and non-technical stakeholders.
- Mentor junior engineers through reviews, pairing, and knowledge sharing while building team capability.
- Produce clear technical documentation as a standard deliverable, including Architecture Decision Records (ADRs),
runbooks
, data dictionaries
, and pipeline lineage notes. - Evaluate and drive adoption of productivity tooling and best practices, setting team norms for safe and effective use.
- Bachelor’s degree in Computer Science, Information Systems, Engineering, or a related technical field.
- 7+ years of hands-on data engineering experience delivering production data pipelines end-to-end.
- Strong command of Py Spark ,
Python
, and SQL
, including Spark optimization, partition management, and query performance tuning in Snowflake
. - Hands-on experience with AWS data services:
S3
, Glue
, EMR
, Aurora
, Lambda
, IAM
, and Cloud Watch
. - Working knowledge of architecture and design principles such as lakehouse patterns and fault-tolerant ingestion
, applied in practice. - Strong analytical and problem-solving skills, including decomposing complex requirements and reasoning through trade-offs.
- Clear communication skills to explain technical decisions and design rationale to engineering peers and business stakeholders.
- Self-motivated learner, interested in staying current with evolving tools, frameworks, and engineering best practices.
Py Spark ,
Python
, AWS EMR
, AWS Glue
, Amazon S3
, AWS Aurora
, AWS Lambda
, IAM
, Cloud Watch
, SQL
, Snowflake
, Snowflake Cortex
, Claude Code
, Claude Code CLI
, GenAI
, Agile/Scrum
, CI/CD
, Architecture Decision Record (ADR).
- Eligible bonus in accordance with the terms of the applicable incentive plan.
- Comprehensive benefits package including health, wellbeing, financial security, and professional development.
- Hybrid flexibility
. - Competitive compensation, with a base salary range of $125,000 to…
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
View / Apply for Jobs
Matching My Jurisdiction