Spark Engineer

Must Have Technical/Functional Skill

We are looking for an experienced Apache Spark Developer with strong expertise in large-scale data processing and performance optimization. The role involves designing, building, and optimizing scalable ETL pipelines using Spark in cloud environments.

Key Skills

• Strong hands-on expertise in Apache Spark 3.5.x or later

• Solid experience with Spark Data Frames, Datasets, and RDDs

• Strong proficiency in Spark SQL

• Experience working in cloud environments (AWS and/or Azure)

• Hands-on experience with cloud storage solutions such as Amazon S3 and Azure Data Lake Storage (ADLS)

• Strong understanding of ETL pipeline design and data modeling

• Proven experience in Spark job performance tuning and query optimization

Preferred / Good to Have

• Experience with Snowflake for data warehousing and analytics

• Exposure to real-time/streaming data processing

• Familiarity with distributed systems and large-scale data architectures

• Scala Programming Language

Roles & Responsibilities

• Design, develop, and maintain large-scale batch and real-time data processing solutions using Apache Spark (v3.5.x or later)

• Work extensively with Spark Data Frames, Datasets, and RDDs for high-volume data processing

• Develop and optimize queries using Spark SQL

• Build and manage ETL pipelines to support analytics and downstream data consumption

• Perform Spark job tuning and performance optimization to improve efficiency and scalability

• Collaborate with data engineers, architects, and analytics teams to deliver end-to-end data solutions

• Ensure data quality, reliability, and adherence to best practices across pipelines

Required Qualifications

• Strong hands on experience with Spark

• In depth knowledge of Apache Spark architecture, internals, and execution model

• Experience working with Spark Data Frames, datasets, RDDs (large scale data processing - batch & real-time)

• Strong SQL skills for data analysis and transformation

• Familiarity with ETL pipelines and data modeling(Snowflake-preferrred)

• Experience with Linux/Unix environments

• Good understanding of distributed computing pri nciples

Good to Have

• Experience with Snowflake for data warehousing and analytics

• Exposure to real-time/streaming data processing

• Familiarity with distributed systems and large-scale data architectures

Salary Range $ 80,000-$100,000 Per year

TCS Employee Benefits

Summary:

Discretionary Annual Incentive.

Comprehensive Medical Coverage:
Medical & Health, Dental & Vision, Disability Planning & Insurance, Pet Insurance Plans.

Family Support:
Maternal & Parental Leaves.

Insurance Options:
Auto & Home Insurance, Identity Theft Protection.

Convenience & Professional Growth:
Commuter Benefits & Certification & amp;
Training Reimbursement.

Time Off:
Vacation, Time Off, Sick Leave & Holidays.

Legal & Financial Assistance:
Legal Assistance, 401K Plan, Performance Bonus, College Fund, Student Loan Refinancing.

#LI-SP1