Founding Backend Engineer

LiteLLM is an open-source LLM Gateway with 18K+ stars on Git Hub and trusted by companies like Rocket Money, Samsara, Lemonade, and Adobe. We’re rapidly expanding and seeking a founding full-stack engineer to help scale the platform. We’re based in San Francisco.

LiteLLM provides an open source Python SDK and Python FastAPI Server that allows calling 100+ LLM APIs (Bedrock, Azure, OpenAI, Vertex

AI, Cohere, Anthropic) in the OpenAI format

We have raised a $1.6M seed round from Y Combinator, Gravity Fund and Pioneer Fund. You can find more information on our website, Github and Technical Documentation.

Companies use LiteLLM Enterprise once they put LiteLLM into production and need enterprise features like Prometheus metrics (production monitoring) and need to give LLM access to a large number of people with SSO (secure sign on) or JWT (JSON Web Tokens).

Your main responsibility will be to ensure that LiteLLM unifies the format for calling LLM APIs in the OpenAI spec. This involves writing transformations to convert API requests from the OpenAI spec to the format of the LLM provider.

You will work closely with our CEO and CTO.

Example projects include:

• Migrating key systems from httpx to aiohttp for 10x higher throughput
• Adding support for Anthropic, Bedrock Anthropic thinking parameter
• Handling provider-specific quirks like OpenAI o-1 streaming support
• Ensuring LiteLLM can compute aggregate spend with 1M+ logs
• Adding cost tracking and logging for /v1/messages Anthropic API

Backend on Python, FastAPI; frontend on JS/TS;
Redis, Postgres, S3, GCS Storage, Datadog, Slack API.

• 1-2 years experience in backend or full-stack development with production-grade systems
• Excited about open source software and engaging with users
• Experience in maintaining and scaling high-performance infrastructure
• Hardworking and able to work in a small, accountable team
• Eager to shape the infrastructure of a rapidly growing tool