Backend AI Platform Engineer

Company Description

Organizations everywhere struggle under the crushing costs and complexities of “solutions” that promise to simplify their lives. To create a better experience for their customers and employees. To help them grow. Software is a choice that can make or break a business. Create better or worse experiences. Propel or throttle growth. Business software has become a blocker instead of ways to get work done.

There’s another option. Freshworks. With a fresh vision for how the world works.

At Freshworks, we build uncomplicated service software that delivers exceptional customer and employee experiences. Our enterprise-grade solutions are powerful, yet easy to use, and quick to deliver results. Our people-first approach to AI eliminates friction, making employees more effective and organizations more productive. Over 72,000 companies, including Bridgestone, New Balance, Nucor, S&P Global, and Sony Music, trust Freshworks’ customer experience (CX) and employee experience (EX) software to fuel customer loyalty and service efficiency.

And, over 4,500 Freshworks employees make this possible, all around the world.

Fresh vision. Real impact. Come build it with us.

We are seeking a Backend AI Platform Engineer to design and build core backend services that power our Agentic AI Platform, enabling reasoning-driven agents, multi-agent orchestration, dialog management, and outcome-based workflows.

You will work closely with the Principal AI Agentic Architect to implement high-performance APIs, orchestration runtimes, agent lifecycle management, and integration with frameworks like Lang Chain, Lang Graph, and Lang Smith. This role is hands-on and ideal for someone passionate about building robust, scalable backend systems in a fast-paced, startup-like environment.

Key Responsibilities:

Platform Backend Development

• Experience building agent runtime systems, orchestration layers, or reasoning engines.
• Implement agent runtime orchestration services for reasoning, planning, and tool invocation (integrating Lang Chain/Lang Graph).
• Develop APIs and SDKs for building, deploying, and managing agent workflows and tools.
• Build stateful dialog management and memory services to support multi-turn, context-aware agents.
• Implement multi-agent communication (A2A) protocols and shared context for collaborative agent behavior.

System Architecture & Scalability

• Work with the Architect to build cloud-native, event-driven backend services using microservices or service mesh architecture.
• Design task scheduling and workflow orchestration (e.g., Temporal/Airflow) for agent planning and long-running operations.
• Develop real-time message buses (Kafka/Pub-Sub) for agent communication and event propagation.
• Optimize LLM orchestration performance and cost efficiency with caching, batching, and token usage monitoring.

Integrations & Data Services

• Implement integrations with enterprise systems, APIs, and knowledge sources (REST, Graph
  
  QL, gRPC).
• Work with vector DBs (e.g., Elastic Search, Pinecone, Weaviate, FAISS) and RAG pipelines for knowledge-augmented reasoning agents.
• Support metadata and telemetry pipelines for Lang Smith-based evaluation and observability.

Security, Compliance & Governance

• Build secure multi-tenant backend services with RBAC, API authentication, and tenant isolation.
• Ensure agent action auditability and reproducibility for compliance and governance.
• Collaborate on implementing platform guardrails (rate limiting, content filtering, safe tool usage).

Testing, Observability & Reliability

• Write automated test suites (unit, integration, load testing) for backend services.
• Implement logging, tracing, and metrics (Open Telemetry, Prometheus, Grafana) for full observability.
• Participate in on-call rotations and improve platform resiliency and fault tolerance.

Please note:
This is a hybrid role that will be based in San Mateo, CA or Bellevue, WA and requires an in-office presence three days per week (Tuesday - Thursday).

Required Qualifications:

• 7+ years of backend engineering experience, with at least 2+ years in AI/ML platform development.
• Strong programming skills in Java and Python
• Hands-on experience…