Remote AI Agent Evaluation Engineer
Jacksonville, Duval County, Florida, 32290, USA
Listed on 2026-10-02
-
Software Development
AI QA / Validation Engineer, AI Reliability/ Performance Engineer, AI Engineer (Applied/Software)
EPAM Systems is seeking an Agent Evaluation Engineer to design and maintain an evaluation framework for AI agents, including automated tests and CI/CD deployment gates. The role focuses on creating multi-layer evaluation suites that blend deterministic checks with LLM-powered graders, simulating multi-turn conversations, and defining reliability metrics.
You will implement staging validations, shadow-mode traffic analysis, and A/B rollout strategies, with feedback loops from production
The following role is for a Remote AI Agent Evaluation Engineer with EPAM Systems Inc.
Are you ready to take on the Remote AI Agent Evaluation Engineer role at EPAM Systems Inc?
We would love to welcome a new Remote AI Agent Evaluation Engineer to our organisation in United States.
For the Remote AI Agent Evaluation Engineer position at EPAM Systems Inc, we are reviewing applications now.
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).