DevOps Engineer - Data Ops

Kai is the AI company rebuilding cybersecurity for the machine-speed era. Founded by second-time founders and trusted by Fortune 500 enterprises, Kai is building a future where security has no categories, no silos, and no human speed bottlenecks. The Kai Agentic AI Platform replaces fragmented, human-limited workflows with agentic AI systems that continuously contextualize, assess, reason, and execute security work at machine speed—making human defenders superhuman.

This role focuses on automating workflows, improving platform reliability, and supporting data engineering teams with efficient development and deployment practices. In a world where digital experiences shape trust and adoption, Data Ops increases development productivity which directly drives product success and customer confidence.

• Design, deploy, and operate scalable data platforms and pipelines, primarily on Azure (Databricks, ADF, ADLS)
• Build, manage, and optimize Apache Spark clusters and workloads for batch and streaming data processing across Azure and AWS environments.
• Implement CI/CD pipelines for data engineering code, Spark jobs, and pipeline configurations using Azure Dev Ops/Git Hub Actions
• Automate infrastructure using Infrastructure as Code (Terraform) and manage containerized workloads with Docker and Kubernetes
• Monitor data pipelines and platforms to ensure data reliability, quality, observability, and cost optimization across Azure and AWS data platforms.
• Enforce security, governance, and best practices, collaborating closely with data engineers and platform teams in Azure-first, multi-cloud environments.

• 6+ years of professional experience in data engineering, Data Dev Ops, or Data platform engineering roles
• Proven experience supporting production-grade data platforms in enterprise environments
• Proven ability to design, build, deploy, and maintain scalable data pipelines (ETL/ELT)
• Deep understanding of Apache Spark for batch and streaming workloads
• Experience creating, configuring, and managing Spark clusters, including performance tuning and cost optimization
• Practical experience with at least one major cloud provider: AWS, Azure, or GCP.
• Strong experience using Terraform for infrastructure automation.
• Proven ability to diagnose and resolve system and infrastructure issues.
• Bonus Skill:
  Experience deploying and managing Spark workloads on Azure Databricks or Azure Synapse

This is a 5‑days a week in‑office role located in our North San Jose, CA location.