Senior Data Engineer

Optomi, in partnership with a leading financial services organization, is seeking a Senior Data Engineer to support enterprise data modernization and large-scale data pipeline initiatives across a growing cloud-based data ecosystem. This role will focus on building and optimizing ETL/ELT pipelines, improving data quality and usability, and supporting governance and secure data‑sharing practices across enterprise platforms. The ideal candidate brings strong experience with SQL, Python, Azure cloud services, and modern data engineering frameworks, along with the ability to work cross‑functionally with business stakeholders, product owners, and technical teams.

• Design, build, and optimize scalable ETL/ELT pipelines supporting enterprise data products and analytics initiatives
• Modernize and support Azure‑based data ecosystems and cloud‑native data workflows
• Develop and maintain SQL, Python, and PySpark‑based data transformation pipelines
• Support data quality, governance, lineage, and secure data‑sharing initiatives across enterprise platforms
• Develop and optimize large‑scale batch and streaming data workflows
• Build and maintain data models, ingestion pipelines, and transformation frameworks
• Support metadata management, cataloging, and critical data element tracking processes
• Implement and support CI/CD automation and Dev Ops workflows for data engineering operations
• Collaborate with business and technical teams to gather requirements and deliver scalable data solutions
• Troubleshoot and optimize pipeline performance, scalability, and operational reliability

• 7+ years of experience in data engineering, ETL development, or data warehousing
• Strong proficiency with SQL and Python
• Experience building and maintaining ETL/ELT pipelines in enterprise environments
• Hands‑on experience with Azure cloud services such as ADF, Synapse, ADLS, or related technologies
• Strong data analysis, troubleshooting, and problem‑solving skills
• Experience supporting CI/CD automation and Dev Ops workflows
• Understanding of data governance, data quality, lineage, and secure data practices
• Excellent communication and cross‑functional collaboration skills

• Experience with Apache Spark, PySpark, or Databricks
• Exposure to Delta Lake, Unity Catalog, ADLS, or Azure Data Factory (ADF)
• Experience supporting enterprise data catalogs, governance frameworks, or metadata management initiatives
• Experience working within regulated enterprise environments such as financial services, healthcare, or insurance
• Familiarity with modern cloud data warehouses and lakehouse architectures