Pyspark Engineer

Job Title:
Pyspark Engineer

Location:
Irving, TX (ONSITE)
F2F Interview
Full Time ONLY

Job Description

Must Have Technical/Functional Skills

PySpark Developer with 5-10 years' experience in data engineering practice. Responsible for designing, developing and maintaining scalable data pipelines, optimizing data workflows and ensuring the integrity and availability of data for business intelligence

Roles & Responsibilities

• Onsite role with strong experience in Apache Spark framework including good understanding of core concepts, performance optimization and industry best practices.

• Proficient in PySpark with hands-on coding experience and ability to implement complex business level transformations.

• Collaborate with the stakeholders and analysts to understand data requirement and deliver robust, creative and innovative solutions.

• Familiarity with unit testing, object-oriented programming (OOPS) concepts and interpreting test results.

• Proficient to write complex and efficient SQL queries to extract the business-critical insights from large-scale data.

• Experience with scheduling of the transformation jobs as per business requirement.

• Perform root-cause analysis and troubleshoot errors on data pipelines, evaluating data quality issues, and implementing corrective fixes