More jobs:
Job Description & How to Apply Below
Senior Principal Engineer - Data Engineerin
g
We will count on you t
o:
As a Data Engineer, you will be responsible for designing and implementing scalable data pipelines a nd AI Based solution using Databrick s. You will handle end-to-end ETL/ELT processes, manage large datasets, an d work with tools like Python, PySpark, and AWS S3 to ensure data is transformed and optimized for analytical use
. You’ll work on cutting-edge cloud and hybrid data projects, transforming raw data into meaningful insights and AI Analytics. You’ll be hands-on from day one, collaborating closely with architects and business stakeholder
s.
What you need to hav
e:
Develop and maintain data pipelines using Databricks and the Medallion Architecture (Bronze, Silver, Gold layer
s).Design AI Based Solution using Databricks Genie and E2E integrati
on.
Knowledge of exposing/consuming Databricks features via API using cloud-native tools or other applicati
on.
Write data transformation scripts using Python and Py Spa
rk.
Store and manage real time data in AWS S3 and integrate with other cloud-based servic
es.
Use SQL to query, clean, and manipulate large datase
ts.
Collaborate with cross-functional teams to ensure data is accessible for business intelligence and analyti
cs.
Monitor and troubleshoot data pipelines for performance and reliabili
ty.
Document data processes and follow best practices for scalability and maintainabili
ty.
Ingest and process structured and unstructured data across batch and streaming sourc
es.
What makes you stand
out?
Experience with Databricks components like :
Pipeline, scheduled / event based job , Genie , Unity Catalog and Datawareh
ouse.
Proficiency in Python, PySpark, and SQL for data processing and transformation using AWS S3
data.
Experience in Data Governance , data access security , and information of configuring Job compute for different Jobs in Data
icks.
Familiarity with version control using
Git.
Understanding of Databricks API and its integration with different Tools and applica
tion.
Bulk data and real time data streaming understan
ding.
Experience with Delta Lake and other Databricks technologies. Knowledge of additional AWS services (e.g., Athena, Glue, Lambda, S3, D
MS ).
Why join our
team:
We help you be your best through professional development opportunities, interesting work and supportive le
aders.
We foster a vibrant and inclusive culture where you can work with talented colleagues to create new solutions and have impact for colleagues, clients and commun
ities.
Our scale enables us to provide a range of career opportunities, as well as benefits and rewards to enhance your well
- being.
Position Requirements
10+ Years
work experience
Note that applications are not being accepted from your jurisdiction for this job currently via this jobsite. Candidate preferences are the decision of the Employer or Recruiting Agent, and are controlled by them alone.
To Search, View & Apply for jobs on this site that accept applications from your location or country, tap here to make a Search:
To Search, View & Apply for jobs on this site that accept applications from your location or country, tap here to make a Search:
Search for further Jobs Here:
×