Software Engineer - Hosted Model Infrastructure

About Palantir

Palantir builds the world’s leading software for data‑driven decisions and operations. By bringing the right data to the people who need it, our platforms empower partners to develop lifesaving drugs, forecast supply chain disruptions, locate missing children, and more.

We are a software engineering team that specializes in enabling machine‑learning models in production. We deploy AI models to run in a variety of environments: air‑gapped government networks, forward‑deployed defense environments, edge nodes, and enterprises with strict data sovereignty requirements. Our customers rely on us for frontier AI capabilities running on hardware they control, often with constrained GPU resources and limited direct access.

Our approach treats models like any other software: continuously tested, continually delivered, packaged for reproducible deployment, and built for long‑term maintainability. You will own services end‑to‑end and work across the full stack, from inference engines and GPU scheduling to deployment pipelines, observability, and integration with the Palantir platform. The goal is to deliver new models and capabilities quickly and continuously.

Join us if you want to solve problems at the intersection of infrastructure and machine learning that directly enable critical customers.

• Back‑end languages:
  Java, Rust, Python, Go
• Model serving engines for GPU‑accelerated inference
• Docker and Kubernetes for containerization and orchestration
• Industry‑standard build tooling (Gradle, Git Hub)

• Build high‑performance model‑serving infrastructure that integrates with security models, hardware constraints, and different inference engines.
• Design intelligent request handling, including authentication, rate limiting, concurrency control, and audit logging for multi‑tenant model access.
• Build and maintain packaging and deployment pipelines that enable fast, secure, and reliable model rollouts across on‑premises and air‑gapped environments.
• Develop observability for production AI systems to enable easy service monitoring and fast incident triage and resolution.
• Debug complex issues and performance problems throughout the stack, including open‑source inference engines, container runtimes, and GPU drivers, in environments you cannot always access directly.
• Design and run testing and benchmarking infrastructure that validates model deployments across varying GPU hardware before they reach production.
• Work with product teams and customers to understand requirements, debug production issues, and deliver the models and capabilities they need.
• Integrate hosted model infrastructure with Palantir’s deployment, configuration, and identity systems.

• Ownership mindset and bias toward quality.
• High empathy for customer needs and drive to deliver reliable, easy‑to‑use models.
• Ability to work effectively across multiple languages and layers of the stack.
• Strong debugging skills and motivation to trace problems from application code through containers, orchestration, and hardware.
• Curiosity about emerging AI capabilities and the ability to evaluate and integrate new models and technologies quickly.
• Active US Security clearance, or eligibility and willingness to obtain a US Security clearance is beneficial, but not necessary.

• 4+ years of professional software engineering experience building and operating production systems.
• Engineering background in Computer Science, Mathematics, Software Engineering, Physics, or a similar field.
• Strong coding skills in programming languages such as Java, C++, Python, Rust, or similar. Familiarity with the Python ML ecosystem is valuable.
• Experience with containers, Kubernetes, and deploying backend services in production environments.
• Strong written and verbal communication skills and ability to iterate quickly with teammates, incorporating feedback and holding a high bar for quality.

The salary range for this position is estimated to be $145,000 – $200,000 per year. Total compensation may also include restricted stock units, a sign‑on bonus, and other incentives. Compensation will be determined by individual…