PySpark Developer at Infosys

Drive big data innovations as a PySpark Developer with Infosys in Mississauga, Ontario. Design and optimize scalable data processing solutions using Apache Spark, Scala, and Hadoop.

Infosys is looking for an experienced PySpark Developer to craft high-performance batch and real-time data pipelines. You will leverage your expertise in Apache Spark and functional programming to collaborate closely with data engineers and stakeholders. This role demands a solid understanding of distributed computing and significant experience with big data technologies.

Key Responsibilities:

• Design and develop data processing pipelines using Apache Spark

• Optimize real-time and batch workflows utilizing Kafka and Hadoop

• Craft complex transformations with Spark SQL and Data Frames

• Implement streaming pipelines with Kafka and Spark Streaming

• Maintain and develop data lake solutions with HDFS and Hive

Requirements:

• 4+ years in Information Technology and Big Data technologies

• Strong expertise in Apache Spark, Scala, and Py Spark

• Experience with Kafka, Hadoop ecosystem, and No

SQL databases

• Familiarity with distributed computing and data processing

• Ability to work in cross-functional teams effectively

Elevate your career by building robust data solutions in a collaborative environment.
#J-18808-Ljbffr