Lead Data Engineer; AWS

Being good neighbors – helping people, investing in our communities, and making the world a better place – is who we are at State Farm. It is at the core of how we operate and the reason for our success. Come join a #1 team and do some good!

HYBRID: Qualified candidates must live or relocate within a 180-mile radius of a hub location listed below and should plan to spend time working from home and some time working in the office as part of our hybrid work environment.
HUB LOCATIONS: Bloomington, IL ;
Richardson, TX ;
Tempe, AZ ;
Dunwoody, GA

SPONSORSHIP: Applicants for this position are required to be eligible to lawfully work in the U.S. immediately; employer will not sponsor applicants for U.S. work authorization (e.g. H-1B visa) for this opportunity.

Innovation Group is seeking a Data Engineer or Lead Data Engineer to join our product execution team within SAIL (State Farm Applied Innovation Lab) focused on Agency and Digital. You will work alongside software engineers, product teams, and business partners to design and build the data capabilities that enable experimentation, product development, and emerging technology solutions.

Our teams operate in environments where the problem, data, and solution may not be fully defined at the outset. We are looking for engineers who are comfortable working through that ambiguity, quickly learning new technologies and domains, and turning complex data needs into practical, scalable solutions.

Day to day, the work centers on three areas: writing code to move and transform data; partnering with software engineers and product teams to determine how data can be accessed, analyzed, and used; and designing and maintaining the platforms where structured and unstructured data lives. Strong engineering fundamentals are essential, but success in this role also requires flexibility, curiosity, innovative thinking, and the ability to navigate a complex organization.

The ideal candidate understands the fundamentals of data engineering and how data solutions are designed, built, deployed, monitored, and maintained as production-quality applications rather than one-off scripts. They communicate clearly, collaborate effectively across technical and non-technical teams, seek to understand the business problem behind the request, and continuously develop their skills as technologies and needs evolve. Depending on the initiative, this role may also help lead data science, machine learning, and generative AI efforts, working across the lifecycle from data exploration and experimentation through engineering, deployment, and ongoing operation.

At the Lead Data Engineer level, the role also includes providing technical leadership, mentoring engineers, influencing architecture and engineering practices, and helping shape the direction of the team's data capabilities and solutions.

• 3 plus years with python development for extracting, transforming, processing, and moving data
• Proficiency writing SQL queries and working with relational databases
• Experience designing, building, and maintaining ETL/ELT data pipelines
• Experience working with structured, semi-structured, and unstructured data, including formats such as CSV, JSON, and text
• Experience with public cloud platforms, preferably AWS, including services such as S3, Glue, and Lambda
• Experience using Git and modern version-control practices
• Familiarity with CI/CD practices and deploying code into shared or production environments
• Comfort working in Linux and command-line environments
• Understanding of software engineering principles such as testing, maintainability, observability, and reusable design
• Ability to troubleshoot data quality, pipeline, integration, and performance issues