Senior Data Engineer

Primary

Skills:

Amazon Redshift,, Secondary

Skills:

Azure Dev Ops,,   Role

Description:

We are looking for a Senior Data Engineer to design, develop, and optimize scalable, reliable, and secure data platforms and pipelines. The role requires strong expertise in modern data engineering, cloud data platforms, data architecture, data modeling, and database technologies, with the ability to translate business requirements into high-quality, production-ready data solutions. The candidate will work closely with cross-functional teams to build and evolve data products, optimize performance and cost, ensure data quality and governance, and contribute to engineering standards and best practices across the organization.

Role Responsibility:
Design, develop, and maintain end-to-end data pipelines, data platforms, and integration solutions across batch and streaming workloads. Translate business, functional, and technical requirements into maintainable, production-ready data solutions. Build data flows across structured, semi-structured, and unstructured sources using appropriate ingestion and integration patterns. Develop data transformation and processing workflows to support large data volumes and evolving business requirements. Implement data quality, validation, reconciliation, testing, and observability practices across data pipelines.

Optimize data pipelines and processing workloads for performance, scalability, and cost efficiency. Implement security, privacy, access control, and data governance throughout the data lifecycle. Monitor pipelines and platform components, troubleshoot failures, perform root-cause analysis, and implement corrective actions. Collaborate with architects, stakeholders, application teams, and QA/UAT teams to deliver solutions aligned with business objectives. Contribute to CI/CD, automation, reusable frameworks, development standards, mentoring, and continuous improvement initiatives.

Role Requirement:
Strong proficiency in SQL, including complex queries, analytical functions, stored procedures, and performance optimization. Strong understanding and hands-on experience with modern data architecture, including Data Lake, Lakehouse, Cloud Data Warehouse, Medallion Architecture, and ELT patterns. Strong understanding of data warehousing and data modeling, including dimensional modeling, fact/dimension design, and Slowly Changing Dimensions (SCD). Hands-on experience with cloud data platforms and modern data engineering technologies.

Experience working with relational and non-relational databases, data ingestion, and streaming technologies. Understanding of data quality, governance, security, and observability practices. Proficiency in Python, Shell, Power Shell, or equivalent scripting and automation technologies.

Experience with Git, CI/CD, and Agile delivery practices. Strong analytical, troubleshooting, and problem-solving skills with the ability to independently diagnose and resolve complex technical issues. Excellent communication, documentation, collaboration, and stakeholder management skills, with a strong sense of ownership and accountability. Additional Requirement:
Excellent hands-on knowledge of dbt architecture, configuration, user setup, and administration.

Experience with enterprise-level dbt installation and configuration. Hands-on experience configuring role-based user access within dbt. Strong experience integrating dbt with Amazon Redshift datasets. Experience integrating dbt with existing Azure Dev Ops (ADO) configurations and enterprise development workflows. Strong understanding of managing authentication and authorization across dbt and underlying Redshift data access. Experience implementing or supporting Natural Language / Conversational BI capabilities through an analytics interface connected to dbt and Redshift.

Ability to ensure conversational analytics access honors the underlying user's credentials, roles, and data access permissions. Strong understanding of security requirements for PII and PCI data, including controlled and authorized access to sensitive datasets. Hands-on dbt development experience, including development and maintenance of transformation models and associated dbt components