SR Azure Data Factory- Remote

Cognizant (NASDAQ: CTSH) is a leading provider of information technology, consulting, and business process outsourcing services, dedicated to helping the world's leading companies build stronger businesses. Headquartered in Teaneck, New Jersey (U.S.). Cognizant is a member of the NASDAQ-100, the S&P 500, the Forbes Global 1000, and the Fortune 500 and we are among the top performing and fastest growing companies in the world.

This role does not support visa-dependent candidates, either now or in the foreseeable future.

Full time

Hybrid

Washington, DC

Azure Data Engineer / Azure Data Architect

Job Description

Location: Hybrid / Remote
Experience: 8+ Years
Employment Type: Full-Time

Position Overview

We are seeking an experienced Azure Data Engineer / Data Architect with strong expertise in designing, building, and optimizing enterprise-scale data platforms within the Microsoft Azure ecosystem. The ideal candidate will have extensive hands‑on experience with Azure Databricks, Azure Data Factory, Azure Data Lake, SQL, Data Warehousing, and Modern Data Architecture patterns
.

This role requires a combination of deep technical expertise, architectural leadership, and hands‑on development capabilities to design scalable, governed, and high-performing data solutions supporting analytics, reporting, AI/ML, and business operations.

Key Responsibilities

Data Architecture & Platform Design

• Design and develop scalable, secure, and reusable enterprise data architectures using Azure cloud technologies.
• Define technical architecture standards, design patterns, and best practices for enterprise data platforms.
• Build scalable Lakehouse architectures utilizing Azure Data Lake Storage and Azure Databricks.
• Design and implement Medallion Architecture (Bronze, Silver, Gold Layers) for modern analytics platforms.
• Create high‑level solution designs, data models, architecture diagrams, and technical documentation.
• Lead architecture reviews and provide technical governance across multiple projects.

Azure Databricks Development (Must Have)

Architecture & Administration

• Design and manage Azure Databricks environments, clusters, compute resources, and work spaces.
• Develop and maintain enterprise‑scale Lakehouse solutions.
• Implement Delta Lake architecture and optimize Delta Table performance.
• Utilize Databricks Unity Catalog for enterprise‑wide data governance and metadata management.
• Configure and manage security, access controls, and data lineage capabilities.

Data Engineering

• Develop PySpark‑based ETL/ELT pipelines for large‑scale structured and semi‑structured datasets.
• Read, transform, validate, and write data using Spark Data Frames and Delta Tables.
• Implement data quality, audit, and reconciliation frameworks using SQL and PySpark.
• Apply Databricks performance tuning and optimization techniques including:
  • Partitioning
  • Z‑Ordering
  • Data Skipping
  • Caching
  • Auto Optimize
  • VACUUM & OPTIMIZE commands

Emerging Technologies

• Stay current with Databricks innovations including:
  • Lakehouse Architecture
  • Lakeflow
  • Delta Live Tables
  • Unity Catalog
  • Databricks Genie
• Exposure to AI/ML workloads and LLM‑based implementations within Databricks environments is highly desirable.

Azure Data Factory (ADF)

• Design and implement enterprise‑grade ADF pipelines and orchestration frameworks.
• Develop dynamic and metadata‑driven data ingestion pipelines.
• Build solutions to migrate data from on‑premise applications and databases to Azure platforms.
• Implement failure handling, monitoring, alerting, and notification frameworks.
• Create parameterized reusable pipeline templates.
• Develop scheduling, dependency management, and workflow orchestration mechanisms.
• Implement Slowly Changing Dimension (SCD) Type 2 solutions using ADF, SQL, and Azure Databricks.

SQL & Data Processing

• Design and optimize complex SQL solutions supporting enterprise reporting and analytics.
• Utilize advanced SQL analytical functions and windowing functions.
• Implement and optimize joins, aggregations, subqueries, Common Table Expressions (CTEs), and Self Joins.
• Perform SQL performance tuning, indexing, query optimization, and execution plan analysis.n>
• Develop reusable stored procedures, views, and database objects supporting enterprise…