Data Platform Engineer

Data Platform Engineer – Pune, India

As we continue our journey, we are building a team of sharp, intelligent innovators who can move fast with great agility and passion. We are currently searching for a Hands‑on Data Platform Engineer to join our high‑performing product and fulfillment center of excellence in Pune, India. If you are passionate about AWS/Azure networking, Palo Alto firewall/router technologies, and thrive in a fast‑paced environment, this opportunity is for you.

• Manage daily operations of Nexusone products on on‑premise and cloud‑based technologies using open‑source Linux‑based data technologies.
• Install, configure, and support CDP clusters in both cloud (AWS, Azure, GCP) and on‑premises environments, ensuring seamless integration and functionality.
• Maintain, patch, and upgrade existing CDP setups, ensuring minimal downtime and adherence to Cloudera’s best practices.
• Evaluate and optimize complex distributed production deployments, identifying bottlenecks and recommending performance enhancements.
• Configure and manage security using tools like Ranger and Kerberos, ensuring robust data protection and compliance.
• Configure Datadog and develop integrations to monitor and alert operations teams.
• Develop and maintain technical documentation, including administration runbooks and knowledge base articles, to support operational excellence and knowledge sharing.
• Work in an agile team and participate in an on‑call rotation, providing expert‑level support and troubleshooting for critical issues.
• Lead and mentor junior team members, fostering a culture of continuous learning and improvement. Collaborate with cross‑functional teams, including data engineers, solution architects, and Dev Ops, to design and implement scalable data solutions.
• Stay current with emerging technologies and industry trends, applying this knowledge to improve the CDP environment and drive innovation.

• In-depth understanding of both on‑premise and cloud network architectures, ensuring seamless integration and efficient data flow.
• Minimum of 5 years of experience in installing and administering the Cloudera Data Platform (CDP), with a proven track record of managing large‑scale deployments.
• Hands‑on experience with open‑source Linux‑based data technologies, including Iceberg, Spark, NiFi, Jupyter Notebooks, Cloudera, Databricks, Kubeflow, MLFlow, and Kafka.
• Proven ability to build and support solutions across major cloud platforms (AWS, Azure, GCP), leveraging cloud‑native tools and open source for optimal performance.
• Strong Cloudera expertise, with a deep understanding of Spark and Airflow for orchestrating complex data workflows.
• Experience integrating Azure Active Directory with FreeIPA and other directory services such as LDAP, enhancing security and user management.
• Up‑to‑date knowledge of the Hadoop Big Data ecosystem, staying current with the latest technologies and best practices.
• Excellent troubleshooting skills, with a thorough understanding of CDP capacity planning, identifying bottlenecks, and optimizing memory utilization, CPU usage, OS performance, storage, and network configurations.
• Strong analytical and problem‑solving abilities, capable of diagnosing and resolving complex technical issues efficiently.

Nice to have

Cloudera Certified Administrator certifications is a plus. Prior experience as a support or development engineer with a focus on Data, Analytics, and AI.

Mid‑Senior level

Full‑time

Engineering and Information Technology

IT Services and IT Consulting