Senior Specialist, Data Engineering

Data Architecture & Modeling

Architect and implement scalable data pipelines for ingestion, transformation, and integration across cloud and on-prem environments, using Azure Databricks, Azure Data Factory (ADF) and medallion architecture principles to support structured, semi-structured, and unstructured data.

Design and maintain conceptual and physical data models aligned with business reporting and analytics needs, ensuring compatibility with bronze, silver, and gold layers of the medallion framework for optimized data quality, performance, and governance.

Collaborate with application development teams to define data flows and ensure smooth integration into production systems, using layered data refinement to support analytics and BI tools.

Performance Optimization & Automation

Optimize performance of data solutions through indexing, partitioning, caching, and parallel processing techniques.

Drive automation and self-healing capabilities in data workflows using orchestration tools and metadata-driven design.

Governance, Security & Compliance

Lead the development of reusable frameworks for data quality, lineage, and governance to ensure trustworthy analytics.

Ensure compliance with data privacy, security, and regulatory standards (e.g., HIPAA, SOC
2).

Analytics Enablement & Reporting

Build and support Power BI solutions to enable data-driven decision-making across the organization.

Develop executive-level dashboards and status reports to communicate key insights and progress.

Leadership & Mentorship

Mentor junior engineers and contribute to technical leadership across the data engineering practice.

Promote best practices in data engineering, Dev Ops, and agile delivery.

Degree or equivalent and typically requires 7+ years of relevant experience.

Extensive hands-on experience in data engineering, including cloud data platforms like Databricks, Snowflake, and Azure Synapse.

Strong proficiency in Python, SQL, and Spark for data processing and transformation.

Deep knowledge of the Microsoft Azure ecosystem and experience with Azure Data Factory and Azure Databricks.

Experience with streaming and batch data processing frameworks using Kafka, Delta Lake.

Proven track record in designing and implementing medallion and lakehouse architectures.

Expertise in data modeling for both transactional and analytical systems.

Familiarity with CI/CD pipelines, version control (Git), and infrastructure-as-code (Terraform, ARM templates).

Excellent communication and stakeholder engagement skills.

Experience with enterprise-scale data integration and modernization programs.

Knowledge of data cataloging and metadata management tools (e.g., Alation).

Exposure to AI/ML pipelines and feature engineering workflows.

Strong problem-solving skills with a focus on performance tuning and cost optimization.

Ability to lead cross-functional initiatives and influence architectural decisions.

Experience in building and presenting executive dashboards and reports using PBI.

Bachelor’s degree or Master’s degree (in Computer Science, Information Systems, or related field) preferred.