Senior Software Engineer II, Data Pipelines

• Act as a technical leader and Subject Matter Expert in data pipeline automation and workflow orchestration
• Design, implement, and maintain reliable data solutions using Airflow, dbt, and Google Cloud data products
• Manage a large portfolio of dbt models using macros and DRY patterns
• Advocate for test-driven development and assist QA in developing a reliable continuous integration and delivery process
• Monitor, troubleshoot, and optimize data pipeline and workflow performance
• Architect scalable solutions and reusable patterns
• Provide implementation, configuration, and deployment documentation
• Proactively address issues and implement innovative solutions
• Participate in agile ceremonies, including daily standups and sprint planning
• Mentor other engineers and foster technical excellence
• Stay current with industry trends and technologies
• Ensure data security, governance, compliance, and privacy standards
• Report to the Director of IT

• Minimum 7+ years of experience in software development
• Extensive experience in Python, SQL, and data pipelines
• Expertise building and automating ETL/ELT pipelines using Airflow and DAG-based workflow management software
• Experience with Airbyte for data ingestion is a plus, but not required
• Deep experience managing dbt models using macros and dbt tests
• Mastery of Python or a comparable scripting language, API integrations, and software architecture
• Deep experience with PostgreSQL and Big Query, including query optimization for performance and cost
• Good understanding of business intelligence tools such as Looker or comparable alternatives
• Experience with Google Cloud Platform, Kubernetes, and Terraform for infrastructure as code
• Proficiency with Parquet, Avro, Hive, and JSONL data formats and data integration techniques
• Experience with Prometheus and Grafana is a plus
• Familiarity with version control systems and CI/CD tools such as Git Hub Actions
• Strong command of agile methodologies, continuous integration, and test-driven development
• Exceptional problem-solving skills and technical leadership
• Ability to influence and guide cross-functional teams and projects
• Must be authorized to work in the U.S.; this position is not eligible for immigration sponsorship

Demonstrates expertise in data pipeline automation and workflow orchestration, with a strong focus on designing and implementing scalable data solutions using Airflow, dbt, and Google Cloud products. Proficient in managing dbt models and advocating for best practices in test-driven development and continuous integration.

• Data Pipeline Automation
• Airflow and DAG-Based Workflow Management
• Python and SQL Proficiency
• Google Cloud Platform Experience
• Technical Leadership and Mentoring

• Python
• SQL
• ETL/ELT Pipeline Automation
• Airflow
• Dbt
• PostgreSQL
• Big Query
• Kubernetes
• Terraform
• Data Integration Techniques

• Problem-Solving
• Technical Leadership
• Influencing Cross-Functional Teams

• Data Security
• Governance
• Compliance
• Agile Methodologies
• Continuous Integration

• Google Cloud Data Products
• Looker
• Prometheus
• Grafana
• Git Hub Actions