Senior Big Data Engineer: LLM Pipelines & Cloud Lakehouse

Our customer is one of the world's largest technology companies based in Silicon Valley with operations all over the world. On this project, we are working with bleeding-edge big data technologies to develop a high-performance data analytics platform, which handles petabytes of data. We are looking for an enthusiastic and technology-proficient Senior Data Engineer, who is eager to participate in design and implementation of a top-notch big data solution that will be deployed at a massive scale.

Responsibilities :

• Build data pipelines to enable training of LLMs
• Work with technical and non-technical stakeholders to collect requirements, integrate with data systems and deliver large-scale datasets
• Operate in a complex multi-cloud data lakehouse, building end-to-end integrated workflows
• Work with privacy, security, and other policy stakeholders to design, review and implement solutions to automate compliant ETL for downstream teams

Requirements :

• Strong understanding of machine learning principles, especially in the context of LLMs.
• 5+ years of experience in big data engineering
• Proficiency in Java, Scala, Spark
• Proficiency in PySpark and Golang(preferred)
• Experience building and leveraging large scale data infrastructure, map-reduce/hadoop , cloud-native deployments
• Experience with alerting, monitoring and remediation automation in a large scale distributed environment
• BS/BA or equivalent degree in computer science or similar (preferred)

What we offer:

• Opportunity to work on cutting-edge projects
• Work with a highly motivated and dedicated team
• Competitive salary
• Flexible schedule
• Full benefits including but not limited to medical, dental, vision
• Corporate social events
• Professional development opportunities

About Us:

Grid Dynamics (Nasdaq:

GDYN) is a digital-native technology services provider that accelerates growth and bolsters competitive advantage for Fortune 1000 companies. Grid Dynamics provides digital transformation consulting and implementation services in omnichannel customer experience, big data analytics, search, artificial intelligence, cloud migration, and application modernization. Grid Dynamics achieves high speed-to-market, quality, and efficiency by using technology accelerators, an agile delivery culture, and its pool of global engineering talent.

Founded in 2006, Grid Dynamics is headquartered in Silicon Valley with offices across the US, UK, Netherlands, Mexico, and Central and Eastern Europe.

To learn more about Grid Dynamics, please visit  Follow us on Facebook, Twitter, and Linked In.