Senior Data Engineer

• Collaborate with data analysts, data scientists, ML engineers, software engineers, and business stakeholders to enable effective use of core data assets
• Design, develop, and maintain scalable data ingestion and transformation pipelines using Python, SQL, and modern data tooling
• Build and optimize data lake, lakehouse, warehouse, and data mart architectures
• Develop and maintain data models including facts, dimensions, feature datasets, and domain-specific data products
• Translate business requirements into design documents (e.g., ERDs, data flow diagrams) data models and ML feature pipelines
• Design and manage cloud-based data and ML infrastructure (Databricks preferred), including development, staging, and production environments
• Design, build, and operationalize machine learning pipelines for training, validation, deployment, and observability (e.g., performance, drift, reliability)
• Support ML model lifecycle management, including versioning, reproducibility, and lineage
• Develop and maintain ML feature stores and reusable feature pipelines for ML models
• Build and integrate AI-powered applications and agentic workflows (e.g., LLM-based agents, retrieval-augmented generation systems, workflow automation agents)
• Design and implement data pipelines for AI systems, including unstructured data (text, logs, embeddings, vector stores)
• Develop and maintain unit, integration, and data quality tests
• Participate in peer code reviews, pull requests, and team coding standards
• Document data pipelines, ML pipelines, models, infrastructure, and standard operating procedures
• Define infrastructure as code and support CI/CD pipelines for data and ML systems
• Ensure data privacy, security, and access control best practices (including AI data governance considerations)
• Identify and implement improvements in efficiency, scalability, resilience, and performance
• Contribute to evolving data, ML, and AI platform architecture, tools, and best practices

• Ability to gather requirements and translate business processes into data, ML, and AI solutions
• Comfortable working cross-functionally with both technical and non-technical stakeholders
• Ability to quickly learn new domains and technologies
• Strong Python development experience
• Advanced SQL development and query optimization skills
• Understanding of Databricks and large-scale data processing
• Experience building and scaling data pipelines using Databricks and Py Spark
• Deep understanding of data lake, lakehouse, data warehouse, and data mart architectures
• Experience with data modeling across a variety of business domains
• Experience with modern data tooling (e.g., dbt or similar transformation frameworks)
• Knowledge of data formats, data patterns, and modeling best practices
• Experience with cloud platforms (AWS preferred)
• Experience with CI/CD pipelines in a data engineering environment
• Git-based development workflows
• Hands‑on experience with AI prompt and agent frameworks (e.g., Claude Code, Cursor, Windsurfer, or similar)
• Experience building AI agents and agentic workflows
• Exposure to LLMs, embeddings, vector databases, or generative AI systems
• Familiarity with handling structured and unstructured data (e.g., text, logs, embeddings)
• Experience building or supporting machine learning pipelines in production
• Familiarity with AI and MLOps in Databricks
• Experience with ML feature engineering and feature stores
• Understanding of ML model lifecycle management, monitoring, and evaluation
• Bachelor's degree in computer science, information systems, a quantitative field, or equivalent practical experience

Demonstrates expertise in designing and developing scalable data ingestion and transformation pipelines using Python and SQL, while effectively collaborating with cross‑functional teams to translate business requirements into actionable data and ML solutions. Proficient in managing cloud‑based data infrastructure and operationalizing machine learning pipelines, ensuring data privacy and governance best practices.

• Python Development
• SQL Development
• Databricks Experience
• Data Pipeline Development
• Machine Learning Lifecycle Management

• Data Ingestion
• Data Transformation
• Data Modeling
• Feature Engineering
• CI/CD Pipelines
• Unit Testing
• Integration Testing
• Data Quality Testing
• AI Prompt Frameworks
• MLOps

• Cross‑Functional Collaboration
• Requirement Gathering
• Adaptability

• Bachelor's Degree in Computer Science
• Information Systems
• Quantitative Field

• Data Lake
• Lakehouse
• Data Warehouse
• Data Mart
• AI Systems
• Unstructured Data
• Generative AI
• Data Governance
• ML Feature Stores
• Data Privacy

• Databricks
• AWS
• Py Spark
• Dbt
• Git