Sr Data Engineer

Job Title:

Data Engineer

Experience:

5-7 years

Relevant Experience – 5 years

Work location:

Bangalore/ Pune/Hyderabad/Chennai/Noida

Job Description

Data engineers design, build, and maintain the infrastructure for an organization's data,

ensuring it is reliable, accessible, and optimized for use by data scientists and analysts. Key

responsibilities include creating and managing data pipelines (ETL processes), optimizing data

storage and processing, ensuring data quality and security, and collaborating with other teams

to meet data needs.

Responsibilities

 Design and build data infrastructure:

Create and maintain scalable data pipelines, databases, and data warehouses to handle large

volumes of data from various sources.

 Ensure data quality and reliability:

Implement data cleansing, validation, and quality checks to ensure the data is accurate and

trustworthy.

 Optimize data systems:

Monitor and tune performance of databases and data processing tasks to ensure efficiency and

speed.

 Collaborate with stakeholders:

Work with data scientists, analysts, and other teams to understand their data requirements and

make data accessible for their use.

Key skills

Data Platform Project

 Autonomous level people

 Skillsets needed (Mandatory)

 Pyspark, Spark SQL
-  Databricks

 Data Governance – Unity Catalog

 Python – Pandera Framework

 REST API Frameworks

Nice to have

 Cloud platforms:

Experience with cloud services (e.g. GCP Services - Cloud build, Big Query,

Firestore etc…

 Airbyte/Fivetran

 Nutter Framework – Experience in writing unit test in Nutter Framework

 Terraform Mandatory.

 CI/CD Concepts

 OAuth 2 Protocol

 Analytical and problem-solving skills:
Strong analytical abilities are needed to troubleshoot

issues and find solutions.