Databricks Data Engineer - Senior - Consulting

Job Title:

Databricks Data Engineer – Senior – Consulting

Location:

Anywhere in Country

Company: EY

At EY, we’re all in to shape your future with confidence.

We’ll help you succeed in a globally connected powerhouse of diverse teams and take your career wherever you want it to go.

We’re looking for a highly skilled Senior Consultant Data Engineer with expertise in cloud data engineering, specifically Databricks.

In this role, you will design and build analytics solutions that deliver significant business value. You will collaborate with other data and analytics professionals, management, and stakeholders to ensure that technical requirements align with business needs. Your responsibilities will include creating scalable data architecture and modeling solutions that support the entire data asset lifecycle.

• Designing, building, and operating scalable on-premises or cloud data architecture.
• Analyzing business requirements and translating them into technical specifications.
• Optimizing data flows for target data platform designs.
• Design, develop, and implement data engineering solutions using Databricks on cloud platforms (AWS, Azure, GCP).
• Collaborate with clients to understand their data needs and provide tailored solutions that meet their business objectives.
• Lead end-to-end data pipeline development, including data ingestion, transformation, and storage.
• Ensure data quality, integrity, and security throughout the data lifecycle.
• Provide technical guidance and mentorship to junior data engineers and team members.
• Communicate effectively with stakeholders, including technical and non-technical audiences, to convey complex data concepts.
• Manage client relationships and expectations, ensuring high levels of satisfaction and engagement.
• Stay updated with the latest trends and technologies in data engineering and cloud computing.

• Strong analytical and decision-making skills.
• Proficiency in cloud computing and data architecture design.
• Experience in data integration and security.
• Ability to manage complex problem-solving scenarios.

• Bachelor’s degree in Computer Science, Engineering or a related field (4-year degree). Master’s preferred.
• 2–4 years relevant experience in data engineering, focusing on cloud data solutions.
• 5+ years of experience in data engineering, focusing on cloud data solutions.
• Expertise in Databricks and experience with Spark for big data processing.
• Proven experience in at least two end-to-end data engineering implementations, including data lake solutions and real-time data pipelines.
• Strong programming skills in Python, Scala, or SQL.
• Experience with data modeling, ETL processes, and data warehousing concepts.
• Excellent problem-solving skills and ability to work independently and as part of a team.
• Strong communication and interpersonal skills, with a focus on client management.

• Strategic thinking: aligning data engineering solutions with business strategies.
• Project management: managing multiple projects simultaneously, ensuring timely delivery.
• Stakeholder engagement: engaging with executives to understand needs and present solutions.
• Change management: guiding clients through change processes related to data transformation and technology adoption.
• Risk management:
  
  identifying potential risks in data projects and developing mitigation strategies.
• Technical leadership: leading technical discussions and making architectural decisions.
• Documentation and reporting: creating comprehensive documentation and reports to communicate project progress.

• Experience with data quality management.
• Familiarity with semantic layers in data architecture.
• Familiarity with cloud platforms (AWS, Azure, GCP) and their data services.
• Knowledge of data governance and compliance standards.
• Experience with machine learning frameworks and tools.

We seek individuals who are not only technically proficient but also possess qualities of top performers. You should be adaptable, collaborative, and driven by a desire to achieve excellence in every project you…