More jobs:
AI Engineer - Agent Platform
Job in
Nashville, Davidson County, Tennessee, 37247, USA
Listed on 2026-07-31
Listing for:
Stellar Consulting Solutions, LLC
Full Time
position Listed on 2026-07-31
Job specializations:
-
Software Development
AI Engineer (Applied/Software), AI Reliability/ Performance Engineer, DevOps, Software Architect
Job Description & How to Apply Below
- Build and deploy AI agents and agentic workflows for real-world business use cases.
- Design and evolve core platform capabilities for agent development, orchestration, and runtime.
- Develop RAG pipelines, tool integrations, and structured reasoning systems.
- Enable flexible usage across multiple LLM providers and model libraries.
- Implement secure, scalable, and observable architectures for AI systems.
- Establish robust CI/CD pipelines for agent lifecycle (build, test, deploy, rollback).
- Define and implement agent testing & evaluation strategies, including:
- Automated evaluation pipelines
- Agentic testing models and simulation-based validation
- Continuous quality, accuracy, and regression testing
- Build and enforce LLM observability and evaluation frameworks (this is core to the role).
- Drive platform standards, reusable components, and engineering best practices.
- Partner across teams and external partners to scale adoption of the platform.
- 8+ years of software engineering experience.
- Strong experience with GCP (Vertex AI, GKE, Big Query, APIs).
- Experience with distributed systems and scalable platform design.
- Hands-on experience with:
- LLMs, embeddings, and RAG architectures
- Agentic workflows and AI-driven systems
- Experience working with multiple model providers / model ecosystems.
- Strong familiarity with agent frameworks such as Lang Chain, Lang Graph, or similar ecosystems.
- Strong experience with:
- CI/CD pipelines and automation for production systems
- Secure system design (authentication, authorization, data access controls)
- API-first and microservices architectures
- Experience designing highly observable systems (metrics, tracing, logging).
- Ability to manage performance, cost, and reliability trade-offs in AI workloads.
- Hands-on experience with LLM evaluation frameworks and observability tooling.
- Model evaluation and benchmarking pipelines
- Test harnesses for agent workflows
- Continuous validation / regression testing for AI systems
- Simulation or scenario-based testing for agent behavior
- Experience with AI governance / Responsible AI practices.
- Familiarity with enterprise-scale AI platforms or shared developer platforms.
- Retail or domain experience (digital, marketing, operations).
- Work on a foundational AI platform initiative with enterprise-wide impact.
- Define how AI agents are built, governed, and scaled across the organization.
- Influence both hands-on engineering and long-term platform direction.
To View & Apply for jobs on this site that accept applications from your location or country, tap the button below to make a Search.
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
Search for further Jobs Here:
×