Big Data engineer

Experience : 4-6yrs

We seek a talented Data Engineer with AI & ML knowledge to join our team. As a Data Engineer or MLOps Engineer, your primary responsibility will be to develop & integrate ML solutions that focus on technology improvements. Specifically, you will be working on projects involving leveraging AI/ML for Data Management Efficiencies & Query Optimizations.

• Collaborate with cross‑functional teams such as Data Scientists, Product Partners and Partner Team Developers to identify opportunities for Big Data, Query (Spark, Hive SQL, Big Query, SQL) tuning opportunities that can be solved using machine learning and generative AI.
• Write clean, high‑performance, high‑quality, maintainable code.
• Create backend applications using Python, Docker, Google Cloud & in‑house ML frameworks to orchestrate end‑to‑end applications.
• Design and develop Big Data Engineering solutions & generative AI applications ensuring scalability, efficiency, and maintainability.
• Implement prompt engineering techniques to fine‑tune and enhance LLMs for better performance and application‑specific needs.
• Stay abreast of the latest advancements in the field of Generative AI application development and actively contribute to the research and development of new Generative AI applications.

• Proven experience working as a Big Data & MLOps Engineer, with a focus on Python, Google Cloud, Spark, Spark SQL, Big Query, and Generative AI applications.
• Deep understanding and experience in tuning Dataproc, Big Query, and Spark applications.
• Solid knowledge of software engineering best practices, including version control systems (e.g., Git), code reviews, and testing methodologies.