Enterprise Data Lake Lead
Listed on 2026-11-05
-
IT/Tech
Data Engineering, Database Administrator, Data Warehousing, Cloud Computing: Infrastructure & Operations
Data Lake Tech. Lead
12 Months
Remote
Primary skills:
Apache Kafka
Databricks platform
Data Hub or Collibra/Alation/Apache Atlas
RDBMS and Modern Data lake
Kubernetes - Functional knowledge
Required Qualifications
Technical Expertise around 12-15 years of experience.
Data Streaming & Processing
5+ years with Apache Kafka (streaming architecture, Kafka Connect, Schema Registry, stream processing)
3+ years with Databricks (Delta Lake, Apache Spark optimization, Unity Catalog, cluster management)
Deep understanding of data lake architecture patterns (Bronze/Silver/Gold, medallion architecture)
Data Governance & Cataloging
3+ years with Data Hub or similar metadata management platforms (Alation, Collibra, Apache Atlas)
Deep experience building and operating enterprise data catalog systems
Expertise in automated metadata extraction, lineage tracking, and impact analysis
Experience with data quality frameworks and metadata-driven data operations
Knowledge of data governance policies, data classification, and compliance automation
Understanding of data discovery, access control, and self-service analytics enablement
Relational Database Systems
5+ years with enterprise RDBMS platforms including:
SQL Server(T-SQL, SSIS, SSRS, replication, Always On Availability Groups)
PostgreSQL(advanced query optimization, partitioning, extensions, streaming replication)
MySQL(replication, clustering, performance tuning)
Strong SQL skills (complex queries, stored procedures, window functions, query optimization)
Database design principles (normalization, indexing strategies, schema design, partitioning)
Change Data Capture (CDC) patterns and implementation (Debezium, Azure Data Factory, AWS DMS, custom solutions)
Database migration experience (schema migration, data migration, zero-downtime migrations)
Cloud & Infrastructure
5+ years with Azure or AWS Cloud including:
Azure:
Data Lake Storage (Gen2), Event Hubs, AKS, Azure SQL, Key Vault, Azure AD, Monitor
AWS: S3, MSK/Kinesis, EKS, RDS/Aurora, Secrets Manager, IAM, Cloud Watch
Cloud-native data services, networking, security, and IAM
3+ years with Kubernetes (deployment strategies, scaling, monitoring, service mesh, Helm)
3+ years with Terraform (modules, state management, multi-environment deployments, multi-cloud)
Programming & Development
Strong proficiency in any programming languages like C#, Go
Expert-levelSQLacross multiple database platforms
Experience with Python for data engineering tasks (preferred)
Familiarity with Shell scripting (Bash)
Leadership & Experience
7+ yearsin data engineering, platform engineering, or database engineering roles
3+ yearsin technical leadership capacity (Tech Lead, Principal Engineer)
Proven track record of delivering large-scale data infrastructure projects
Experience leading teams of 5-10+ engineers
Strong architectural design and system thinking capabilities
Experience migrating legacy RDBMS workloads to modern data lake architectures
Demonstrated ability to balance technical excellence with business needs
Regards,
Ram
Celersoft LLC
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).