Technical Solutions Engineer; Apache SparkTM Bengaluru, India

Position: Staff Technical Solutions Engineer (Apache SparkTM) Bengaluru, India
Staff Technical Solutions Engineer (Apache Spark™)

Bengaluru, India

Overview

As a Spark Staff Technical Solutions Engineer, you will provide deep‑dive technical and consulting‑related solutions for challenging Apache Spark™, ML, AI, Delta, Streaming, and Lakehouse issues reported by our customers and resolve any challenges involving the Databricks unified analytics platform with your highly comprehensive technical and customer communication skills. You will assist our customers in their Databricks journey and provide guidance, knowledge, and expertise that they need to realize value and achieve their strategic objectives.

Outcomes

• Perform initial level analysis and troubleshooting of Apache Spark™ issues using Spark UI metrics, DAG, event logs for job slowness issues.

• Troubleshoot, resolve, and suggest deep code‑level analysis of Spark to address issues related to Spark core internals, Spark SQL, Structured Streaming, Delta, Lakehouse and other Databricks runtime features.

• Assist customers in setting up reproducible Spark problems with solutions in Spark SQL, Delta, memory management, performance tuning, streaming, data science, and data integration.

• Contribute to the development of tools and automation initiatives.

• Participate in the Designated Solutions Engineer program and drive day‑to‑day Spark and cloud issues for strategic customers.

• Provide best‑practice guidance on Spark runtime performance and use of Spark core libraries and APIs for custom‑built solutions.

• Provide front‑line support on third‑party integrations with the Databricks environment.

• Plan and coordinate with Account Executives, Customer Success Engineers, and Resident Solution Architects to address customer issues.

• Participate in screen‑sharing meetings, Slack conversations, and drive major Spark issues at an individual contributor level.

• Review Engineering JIRA tickets and proactively brief support leadership on action items.

• Manage assigned Spark cases daily and adhere to committed SLAs.

• Build internal wiki and knowledge base articles for the support team and customers.

• Achieve expectations of support organization KPIs.

• Coordinate with Engineering and Backline Support teams to identify and report product defects.

• Be a true proponent of customer advocacy.

• Participate in weekend and weekday on‑call rotation, run escalations during outages, and handle critical operational issues.

• Strengthen AWS/Azure and Databricks platform expertise through continuous learning.

Competencies

• 10+ years of experience designing, building, testing, and maintaining Python/Java/Scala applications in delivery and consulting environments.

• 6+ years of hands‑on experience in at least two or more Big Data, Hadoop, Apache Spark™, Machine Learning, AI, Streaming, Kafka, or Data Science use cases at production scale. Spark experience is mandatory.

• Hands‑on experience in performance tuning/troubleshooting of Hive and Spark applications at production scale.

• Proven real‑time experience in JVM and memory‑management techniques such as garbage collection, heap/thread‑dump analysis.

• Working knowledge of data lakes and SCD‑type use cases at production scale.

• Hands‑on experience with SQL‑based databases, data‑warehousing/ETL technologies such as Informatica, Data Stage, Oracle, Teradata, SQL Server, or MySQL.

• Linux/Unix administration skills is a plus.

• Hands‑on experience with AWS, Azure, or GCP is preferred.

• Excellent written and oral communication skills.

• Demonstrated analytical and problem‑solving skills in a distributed big‑data computing environment.

About Databricks

Databricks is the data and AI company. More than 10,000 organizations worldwide — including Comcast, Condé Nast, Grammarly, and over 50 % of the Fortune 500 — rely on the Databricks Data Intelligence Platform to unify and democratize data, analytics, and AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache Spark™, Delta Lake, and MLflow.

Benefits

At Databricks, we strive to provide comprehensive benefits and perks that meet the needs of all of our employees. For specific details on the…