Principal Machine Learning Engineer

At Palo Alto Networks, we’re united by a shared mission—to protect our digital way of life. We thrive at the intersection of innovation and impact, solving real-world problems with cutting‑edge technology and bold thinking. Here, everyone has a voice, and every idea counts. If you’re ready to do the most meaningful work of your career alongside people who are just as passionate as you are, you’re in the right place.

In order to be the cybersecurity partner of choice, we must trailblaze the path and shape the future of our industry. This is something our employees work at each day and is defined by our values:
Disruption, Collaboration, Execution, Integrity, and Inclusion. We weave AI into the fabric of everything we do and use it to augment the impact every individual can have. If you are passionate about solving real-world problems and ideating beside the best and the brightest, we invite you to join us! We believe collaboration thrives in person. That’s why most of our teams work from the office full time, with flexibility when it’s needed.

This model supports real‑time problem‑solving, stronger relationships, and the kind of precision that drives great outcomes.

The AI Canvas team is building our next‑generation data exploration experience for cybersecurity. AI Canvas is designed to transform how security teams explore, understand, and act on complex security data by combining natural language interaction, real‑time visualization, dashboards, and intelligent agents in a unified experience. We are building beyond traditional dashboards and static user interfaces. AI Canvas is based on a modern agent‑ and skill‑driven architecture, bringing together AI, data exploration, visualization, and collaboration.

We are looking for engineers who are comfortable operating in ambiguity, excited by hard technical problems, and motivated to build reliable AI systems that security teams can trust. As a Principal Machine Learning Engineer on the AI Canvas team, you will take significant technical ownership of the AI layer powering the platform. You will help define the architecture, build production‑grade AI systems, and shape how intelligence is integrated throughout the product.

You will work across large language models (LLMs), retrieval‑augmented generation (RAG), AI agents and assistants, agent harnesses, natural‑language‑to‑query generation, evaluation systems, guardrails, and cloud‑based AI infrastructure. This is a highly technical and hands‑on role. You will work closely with ML, backend, UI, product, and design teams to solve challenging AI problems in cybersecurity, where reliability, accuracy, scalability, latency, observability, and trust are critical.

• Provide technical leadership for the architecture and development of the AI capabilities powering AI Canvas, from early design through production deployment and continuous improvement.
• Design and build scalable, production‑grade systems using LLMs, RAG, AI agents, assistants, tools, skills, and agent harnesses.
• Architect intelligent workflows that enable users to explore complex security data through natural language, including natural‑language‑to‑query generation, follow‑up interactions, clarification, investigation, and troubleshooting.
• Define and evolve the architecture for agentic AI systems, including orchestration, context management, tool invocation, memory, reasoning workflows, and multi‑step task execution.
• Build robust evaluation frameworks and harnesses for measuring AI quality, including correctness, relevance, reliability, regression detection, and end‑to‑end product behavior.
• Establish evaluation methodologies using automated metrics, LLM‑based evaluators, human evaluation,…