Sr. Software Engineer

Spectro Cloud helps platform teams and cloud providers modernize and manage infrastructure for the AI era without adding tools or operational complexity. PaletteAI gives enterprises, public sector organizations, and service providers one platform to build, deploy, manage, and scale full-stack environments across virtual machines, Kubernetes, edge, and AI infrastructure. PaletteAI’s Inference Launchpad extends it to local model serving, with open models running on day one.

Spectro Cloud helps platform teams and cloud providers modernize and manage infrastructure for the AI era without adding tools or operational complexity. PaletteAI gives enterprises, public sector organizations, and service providers one platform to build, deploy, manage, and scale full-stack environments across virtual machines, Kubernetes, edge, and AI infrastructure. PaletteAI’s Inference Launchpad extends it to local model serving, with open models running on day one.

Founded by the team behind Palette, a leading Kubernetes management platform, and backed by leading investors, Spectro Cloud serves customers worldwide, including regulated and air-gapped environments.

At Spectro Cloud, we are looking for a senior engineer to own AI inference  customers run models on infrastructure they control, from data centers to edge sites. You will build the systems that serve those models in production and keep them fast, reliable, and efficient. You will bring new models into service, improve how they perform under real demand, and set the standard for how we test and release them.

Your work will shape how our customers use AI in production. We are looking for an engineer who enjoys difficult problems, works well across teams, and can turn a deep technical issue into a clear plan. Experience with Kubernetes is highly desired.

As a software engineer at Spectro Cloud, you will succeed by embracing adaptability, tackling complex challenges, and navigating dynamic environments with confidence. You can break down intricate problems into manageable steps, deliver value iteratively, and explain your work clearly to the people who depend on it. You take ownership of outcomes and see work through to production. At Spectro Cloud, we champion innovation and collaboration, fostering a culture where shared ideas drive progress.

• Build and operate AI inference services in production, with engines such as vLLM, SGLang, or TensorRT-LLM
• Bring new models into service, from first test to full rollout
• Improve throughput, response time, and memory use across the stack
• Find the root cause of performance and reliability issues, then fix them
• Design benchmarks and release criteria for every model we ship
• Set reliability targets and meet them in production
• Scale inference across many GPUs and nodes
• Run inference on modern GPU platforms, including the latest accelerators
• Automate deployment and operations on Kubernetes
• Write runbooks, tuning guides, and clear technical documentation

While we don't expect candidates to check every box, we do look for the following when assessing applicants:

• Hands-on experience running AI or LLM inference in production, with an engine such as vLLM or TensorRT-LLM
• A working understanding of inference engines, including batching, caching, and memory management
• Experience improving the performance, reliability, and efficiency of production systems
• Experience with distributed systems and multi-GPU workloads
• Experience designing benchmarks and release criteria
• Bachelor's degree in Computer Science or a related field
• 4+ years of software development experience, or 2+ years with a Master's degree
• Strong Python and hands-on Py Torch
• Golang for backend services and APIs
• Wor…