Principal Databricks Data Engineer

Overview

Inclusion without Exception

Tata Consultancy Services (TCS) is an equal opportunity employer and embraces diversity in race, nationality, ethnicity, gender, age, physical ability, neurodiversity, and sexual orientation to create a workforce that reflects the societies we operate in. Our commitment to culture and diversity is reflected in our people stories and implemented through equitable workplace policies and processes. TCS is the technology partner of choice for industry-leading organizations worldwide, with a focus on innovation, engineering excellence, and customer service.

The company aims to become the world s largest AI-led technology services company and supports clients across the full AI stack, from infrastructure to intelligence.

Rooted in the Tata Group heritage, TCS is focused on creating long-term value for clients, investors, employees, and the community. With a global footprint of 56 countries and 194 service delivery centers, TCS has been recognized as a top employer across continents and partners with clients to navigate multiple technology cycles.

• Senior level data engineering experience
• Experience in enterprise Data Warehouse and Data Lake platforms
• Hands-on experience with Databricks and Spark at scale
• Strong experience in modernizing legacy Cloudera platforms (CDH/CDP, Hive, HBase, Impala, Spark) to Databricks Lakehouse
• Redesign ingestion, transformation, and consumption patterns from HDFS-based architecture to cloud object storage and Delta Lake
• Refactor legacy Hive/Impala logic into PySpark and Spark SQL ELT pipelines
• Ensure data reconciliation, audit integrity, and consistency during migration
• Design and govern enterprise Data Warehouse and Data Lake/Lakehouse architectures
• Implement layered architecture including Raw/Landing, Curated/Conformed, and Semantic/Consumption layers
• Modernize traditional EDW platforms into scalable lakehouse architectures
• Strong experience in finance and risk data models including General Ledger, Sub-ledger, financial hierarchies, and risk exposure models (credit, liquidity, market risk)
• Enable reporting use cases including aggregation, drill-down, and drill-back capabilities
• Build and manage semantic/consumption layers for BI, reporting, and analytics
• Define business metrics, dimensions, hierarchies, and KPIs
• Experience with Databricks SQL, Delta tables, and dbt or similar frameworks
• Develop and optimize large-scale data pipelines using PySpark, Spark SQL, and Delta Lake
• Implement Medallion architecture (Bronze, Silver, Gold layers)
• Optimize workloads using Z-ORDER, OPTIMIZE, caching, and cluster configurations
• Implement data governance, data quality frameworks, reconciliation controls, and exception handling
• Establish data lineage and metadata management
• Ensure data security, access control, and compliance standards
• Experience with cloud platforms such as AWS or Azure
• Experience with CI/CD pipelines using Git, Terraform, Jenkins, or Azure Dev Ops
• Familiarity with orchestration tools such as Airflow or Databricks Workflows
• Experience with dbt is a plus
• Act as a technical authority and lead architecture decisions
• Mentor and guide senior engineers and establish engineering standards
• Solid stakeholder management with finance, risk, analytics, and governance teams
• Ability to translate complex data structures into business-ready insights

• Experience in BFSI, Capital Markets, or regulatory reporting
• Exposure to SAP Finance, Oracle Financials, or S/4
  
  HANA
• Experience supporting AI/ML workloads
• Databricks or cloud certifications

• Lead Cloudera to Databricks transformation initiatives
• Shape enterprise finance and risk data platforms
• Support regulatory, management, and analytical reporting systems

CA $ 100,000 - CA $ 120,000 Per Year

TCS is committed to meeting accessibility needs in accordance with the Accessibility for Ontarians with Disabilities Act (AODA) and the Ontario Human Rights Code (OHRC). If accommodations are required during recruitment and selection, please inform Human Resources.