Technical Lead, Spark

Business Area:
Engineering | Seniority Level: Mid-Senior Level

Senior Staff Software Engineer – Spark (Java)

Cloudera empowers organizations to transform complex data into clear insights. This role focuses on the Cloudera distribution of Apache Spark and Livy, building enterprise‑grade systems that run on thousands of nodes and process petabytes of data.

• Design and prototype new features for Cloudera’s data engineering experience, leading teams to production at scale.
• Contribute to core open‑source projects such as Apache Spark and Livy.
• Develop in Scala, Java, or Python on modern platforms.
• Gain expertise in distributed data processing, from SQL planners and optimizers to data layout and table formats (Parquet, Iceberg) and fault tolerance.
• Work on large‑scale distributed systems in production clusters, debugging deployment issues and performing root cause analysis.
• Improve internal infrastructure and collaborate with stakeholders across the organization.

• B.Sc. or M.Sc. in a related field or equivalent professional experience.
• Minimum of 6 years professional software development experience.
• Strong leadership in delivering complex product enhancements.
• Proficiency in Java, Scala, or Python (experience with at least one of these languages). Willingness to learn additional languages is expected.
• Experience with systems design, distributed systems, and clean coding practices.
• Excellent oral and written communication skills.
• Self‑motivated problem solver capable of working independently.
• Open‑mind attitude, eagerness to learn, and a passion for building high‑quality products.

• SQL planner and optimizer experience.
• Hands‑on work with Apache Spark, Livy, or related technologies.
• Design and development of large‑scale distributed systems with a focus on scaling, performance, and scheduling.
• Experience with at least one cloud provider (AWS, Azure, GCP).

• Generous PTO and Unplugged Days.
• Flexible WFH policy.
• Mental and physical wellness programs.
• Phone and Internet reimbursement.
• Access to continuous career development.
• Comprehensive benefits package and competitive compensation.
• Paid volunteer time.
• Employee Resource Groups.

Relocation is not eligible, and the role does not include immigration sponsorship. Hybrid, remote, or on‑site work options are available.

EEO/VEVRAA:
Cloudera is an equal opportunity employer. All qualified applicants will receive consideration for employment without respect to race, color, religion, sex, national origin, age, disability, or other protected status.