Senior Data Engineer

• Lead development of robust, observable, and measurable applications using XP practices and a user-centric approach
• Participate in the full application lifecycle with designers, product managers, and engineers
• Use critical thinking, experimentation, data, and industry best practices to achieve business outcomes
• Facilitate group discussions and team ceremonies
• Establish and lead product engineering and software standards
• Ideate products from user problem spaces through ranked, testable solutions
• Research current technology trends and practices
• Lead technical initiatives across the team and department
• Develop, automate, and maintain batch and streaming ETL pipelines
• Build and manage cloud-based data ecosystems on GCP
• Design and optimize data models for data lakes and warehouses
• Implement real-time ingestion and streaming data processing
• Optimize data performance, scalability, and cost efficiency across GCP
• Ensure PCI and PII data compliance
• Integrate GenAI tools for data quality and analytics enhancement
• Collaborate with stakeholders, data scientists, and full-stack engineers to deliver trusted, documented, reusable data products
• Perform additional assigned tasks

• 4+ years of experience in software development
• Understanding of application design patterns, event-driven architecture, databases, schemas, and testing strategies
• In-depth knowledge and experience with continuous integration, continuous deployment, and test-driven development
• Bachelor's Degree or equivalent in MIS, Computer Science, or a related field (preferred)
• Experience with large-scale application troubleshooting and performance tuning (preferred)
  Exposure to major cloud platforms: GCP, AWS, or Azure (preferred)
• Familiarity and experience with XP (Extreme Programming) (preferred)
• Experience with Apache Airflow, Apache Spark, Python, and Scala for ETL pipelines
• Experience with GCP services including Big Query, Bigtable, Dataproc, Pub/Sub, Cloud Storage, IAM, and VPC
• Experience designing SQL and No
  
  SQL data models using Big Query, MongoDB, and Snowflake
• Experience writing complex SQL queries for data transformation, aggregation, and analytics optimization
• Experience applying TDD to Airflow workflows, Spark jobs, and transformation logic
• Knowledge of data mesh and data-as-a-product principles
• Experience with Kafka Connect and streaming technologies such as Spark Streaming or Apache Flink
• Knowledge of GCP data performance, scalability, and cost optimization
• Knowledge of PCI, PII, GDPR, PCI DSS, SOX, and CCPA compliance
• Experience integrating GenAI tools such as OpenAI, Gemini, and Anthropic LLMs
• Ability to collaborate with stakeholders, data scientists, and full-stack engineers

Demonstrates expertise in developing and optimizing cloud-based data ecosystems on GCP, utilizing ETL pipelines and real-time data processing. Proficient in applying XP practices, data compliance standards, and collaboration with cross-functional teams to deliver high-quality data products.

• GCP Cloud Services
• ETL Pipeline Development
• Data Compliance (PCI, PII, GDPR)
• Apache Airflow and Spark
• Test-Driven Development (TDD)

• Software Development
• Data Modeling (SQL, No
  
  SQL)
• Continuous Integration and Deployment
• Event-Driven Architecture
• Performance Tuning
• Apache Kafka
• Data Mesh Principles
• GenAI Tool Integration
• Streaming Data Processing
• Complex SQL Query Writing

• Critical Thinking
• Collaboration
• Facilitation
• User-Centric Approach
• Experimentation

• Bachelor's Degree in MIS, Computer Science, or Related Field

• Data Lakes
• Data Warehouses
• Data Quality
• Data Performance Optimization
• Cloud Platforms (AWS, Azure)

• GCP (Big Query, Bigtable, Dataproc, Pub/Sub, Cloud Storage)
• Apache Spark
• Apache Airflow
• Python
• Scala