Microsoft Azure Data engineer Cary, NC local

Microsoft Azure Data engineer
Cary, NC local only
Contract

Design and build batch and streaming data pipelines using Azure, Databricks, and Kafka.

• Develop ETL/ELT solutions using Spark, SQL and Python/Pyspark, or Scala
• Implement and support Data Lake / Lakehouse architectures
• Integrate data from legacy and modern platforms. Preferred postgres DB migration to Azure Cloud.
• Expertise in handling Data migration using API’s.
• Apply data quality checks and support data governance basics (metadata, lineage, access controls)
• Optimize Spark jobs for performance, reliability, and cost
• Collaborate with cross-functional teams, Product owners, Admin teams, BSAs, Scrum & Project managers

• 8+ years of experience in data engineering in azure using Azure Databricks.
• Strong experience with Azure Data Lake, Databricks, ADF, Postgres.
• Strong Data Engineering experience in Big Data (Cloudera)
• Hands-on expertise in SQL, Python and PySpark or Scala
• Proven experience with batch & real-time ingestion
• Experience with Kafka and big data processing

• Informatica Power Center
• Informatica IDMC or equivalent cloud data integration tools
• SQL Server, Oracle, Azure SQL
• Cosmos DB
• Unix/Linux
• Maestro (or similar enterprise schedulers)
• Data tools such as Precisely or equivalent

• Insurance domain knowledge (Policy, Claims, Billing, Underwriting, Actuarial, Regulatory data)
• Experience with Azure Dev Ops (AzDO) and Git Hub-based CI/CD
• Practical usage of Copilot for data engineering and development productivity
• Exposure to Generative AI / GenAI use cases in data platforms and analytics

Microsoft Azure (ADB/ADF)

Microsoft Azure (ADB/ADF)