Senior Data Engineer

The Senior Data Engineer will design, develop, and optimize scalable cloud-based data platforms that support enterprise analytics, regulatory reporting, business intelligence, and machine learning initiatives. This role is responsible for building high-performance ETL/ELT pipelines, implementing data engineering best practices, and delivering reliable, high-quality datasets across cloud environments. The ideal candidate will have strong expertise in Python, SQL, cloud data platforms, distributed data processing, and modern data engineering technologies.

• Design, develop, and maintain scalable ETL/ELT pipelines to ingest data from multiple internal and external sources.
• Build and optimize cloud-based data pipelines using modern data engineering frameworks and orchestration tools.
• Develop data transformation workflows for structured, semi-structured, and unstructured data.
• Design, develop, and maintain data models supporting analytics, reporting, and machine learning initiatives.
• Collaborate with data architects, business analysts, data scientists, and cross-functional teams to deliver high-quality data solutions.
• Implement data quality validation, metadata management, and data governance standards.
• Optimize SQL queries and distributed processing jobs for performance and scalability.
• Develop reusable data engineering frameworks, automation scripts, and best practices.
• Support batch and near real-time data processing requirements.
• Monitor production data pipelines, troubleshoot failures, and participate in production support and on-call rotations.
• Participate in Agile ceremonies, architecture discussions, code reviews, and technical design sessions.
• Ensure compliance with enterprise security, privacy, and regulatory standards.
• Create and maintain technical documentation for data pipelines, processes, and architecture.

• Bachelor's degree in Computer Science, Information Systems, or a related field.
• 6+ years of experience in Data Engineering or ETL development.
• Strong SQL skills with experience designing, optimizing, and troubleshooting complex queries.
• Hands-on experience with Python for data processing, automation, and pipeline development.
• Experience with cloud platforms such as AWS, Azure, or GCP.
• Experience with ETL tools such as Informatica, Talend, Data Stage, AWS Glue, Azure Data Factory, or similar technologies.
• Experience working with analytical databases such as Snowflake, Amazon Redshift, Azure Synapse, Big Query, or equivalent.
• Experience with Apache Spark or PySpark for distributed data processing.
• Experience using Git and CI/CD practices for source control and deployment automation.
• Strong understanding of data warehousing concepts, dimensional modeling, and data architecture.
• Strong analytical, troubleshooting, and problem-solving skills.
• Excellent communication and collaboration skills.

• Experience with Apache Airflow, Control-M, or other workflow orchestration platforms.
• Experience with Kafka or other streaming technologies.
• Experience with Databricks and Delta Lake.
• Experience working in financial services, banking, insurance, risk management, or related industries.
• Familiarity with data governance, data lineage, and metadata management tools.
• Experience supporting machine learning or AI data pipelines.
• Experience working in Agile/Scrum development environments.