Data Engineer

Mandatory Skills: Python, DBT, SQL, PySpark, and Microsoft Fabric.

Job Description:
Data Engineering & Pipelines
- Design, build, and maintain scalable data pipelines using Python, SQL, PySpark, and Microsoft Fabric.
- Integrate with external systems using REST/Graph APIs, managing authentication, pagination, rate limits, and error handling.
- Support and enhance existing pipelines, ensuring high reliability, observability, and performance.
- Migrate current Python-based or legacy pipelines into Microsoft Fabric (Data Factory, Data Engineering/Notebooks, Lakehouse).
Data Analytics & Modeling
- Build and maintain data models, semantic layers, and analytics-ready datasets.
- Develop transformations and calculations needed for BI and reporting.
- Create or support analytical outputs using tools such as Power BI.
- Implement data quality checks, validation rules, and data governance best practices.
Operations & Continuous Improvement
- Troubleshoot pipeline/data issues, identify root causes, and provide long-term solutions.
- Optimize performance, cost, and reliability across all data workflows.
- Collaborate with business, product, analytics, and IT teams to translate requirements into technical solutions.
- Maintain documentation, data lineage, and change management practices.