AI Engineer | Agentic AI, AWS Bedrock AgentCore & Langfuse
Listed on 2026-11-04
-
Software Development
AI Engineer (Applied/Software), AI QA / Validation Engineer, Python, AI Reliability/ Performance Engineer
- Contract
Job Summary
We are seeking a hands‑on AI Engineer with expertise in Python, Agentic AI, AWS Bedrock Agent Core, and LLM evaluation to build, deploy, monitor, and continuously improve production‑grade AI agents.
Key Responsibilities
Develop AI agents using Python and Claude Agent SDK.
Deploy agents using AWS Bedrock Agent Core Runtime.
Design golden datasets and implement LLM-as-a-Judge evaluations.
Integrate automated evaluations into CI/CD pipelines as quality gates.
Implement Langfuse/Open Telemetry for AI observability and tracing.
Evaluate agent accuracy, task completion, tool execution, and performance.
Support RAG pipelines and AWS data processing workflows.
Required Skills
Python and Agentic AI development.
AWS Bedrock Agent Core Evaluations / Bedrock evaluation.
Golden-dataset design and curation.
LLM-as-a-Judge design and calibration against human labels.
Automated evaluation and CI gate integration.
Langfuse and/or Open Telemetry.
Good to Have
RAGAS or equivalent evaluation frameworks.
Trajectory evaluation and agentic red‑teaming.
Bedrock Guardrails, Strands Agents SDK, and Git Hub Actions.
Qualifications
6-8+ years of software engineering, cloud engineering, or AI/ML experience.
Hands‑on experience with production AI agents and continuous evaluation.
Strong problem‑solving and troubleshooting skills.
All your information will be kept confidential according to EEO guidelines.
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).