Big Data Engineer

Overview

Join to apply for the Big Data Engineer role at Tata Consultancy Services
.

• Design, develop, and maintain scalable ETL/ELT pipelines using PySpark, Airflow, and GCP-native tools.
• Build and optimize data warehouses and analytics solutions in Big Query.
• Implement and manage workflow orchestration with Airflow/Cloud Composer.
• Write complex SQL queries for data transformations, analytics, and performance optimization.
• Ensure data reliability, security, and governance across pipelines.
• Conduct performance tuning and cost optimization of Big Query and PySpark workloads.
• Collaborate with analysts and product teams to deliver reliable data solutions.
• Troubleshoot, debug, and resolve production issues in large-scale data pipelines.
• Contribute to best practices, reusable frameworks, and automation for data engineering.
• Develop and maintain data pipelines using Big Data processes with focus on ingesting, storing, processing, and analyzing large datasets.
• Able to design, code, test, maintain, and document application components and communicate complex data structures.
• Lead reviews of colleagues  work and define test conditions based on requirements.
• Demonstrate deep understanding of core tools used in planning, analyzing, crafting, building, testing, configuring, and maintaining applications.
• Monitor system performance and availability and improve software quality through root cause analysis.

• 5+ years of experience in Data Engineering/Data Warehousing using Big Data technologies (preferred).
• Expertise in distributed ecosystems and Python programming.
• Strong knowledge of Hadoop and Spark architecture and operations.
• Hands-on experience with writing and understanding complex SQL (Hive/PySpark Data Frames) and optimizing joins on large datasets.
• Experience with UNIX shell scripting; ability to design and develop optimized data pipelines for batch and real-time processing.
• Ability to analyze, design, develop, test, and implement system applications; ability to document technical specifications.
• Able to communicate and document technical and functional aspects effectively.

• Experience
  
  Required:
  
  5+ Years
• Employment type:
  
  Full-time
• Location:
  
  Phoenix, AZ (local candidates preferred)
• Salary Range: $90,000 to $110,000 per year

• TCS Employee Benefits
  
  Summary:
  
  Discretionary Annual Incentive, Comprehensive Medical Coverage, Family Support (maternal & parental leaves), Insurance Options, Certification & Training Reimbursement, Paid Time Off, and other core benefits.