ZR__JOB

Jacksonville, United States | Posted on 10/22/2026

The key is time zone coverage and so we can expand our search across the West Coast - remote/hybrid but necessarily located in the western time zones. You must be comfortable to travel to the client site for workshops/meetings.

Overall experience: 12-20+ years Cloud Platform: AWS Primary Language:
Python

• AI Engineer (Agentic AI / AWS Bedrock)

Experience: 6–8+ Years.
Role: AI Engineer / Generative AI Engineer.
Location: USA.
Specialization: Agentic AI, AWS Bedrock Agent Core, Claude Agent SDK, Langfuse, AI Observability & Evaluation.

We are seeking an experienced AI Engineer with strong hands-on expertise in building, deploying, and monitoring production-grade AI agents and Generative AI solutions using AWS technologies.

The ideal candidate will have experience developing AI agents using Claude Agent SDK
, deploying them on Amazon Bedrock Agent Core Runtime
, automating deployments through Git Hub Actions
, and implementing Langfuse for observability and continuous evaluation of agent traffic
.

The role requires a strong combination of AI engineering, cloud deployment, data pipeline development, and LLM performance monitoring.

• Design, develop, and deploy AI agents using Claude Agent SDK and Python.
• Build intelligent, autonomous agents capable of executing complex workflows and interacting with external tools, APIs, and knowledge bases.
• Develop agent orchestration workflows with appropriate error handling, retry mechanisms, and guardrails.
• Integrate AI agents with enterprise applications and data sources.

• Deploy and manage AI agents using Amazon Bedrock Agent Core Runtime.
• Develop scalable, secure, and production-ready agent architectures on AWS.
• Integrate agents with Amazon Bedrock foundation models and enterprise knowledge bases.
• Implement agent authentication, access controls, logging, and runtime monitoring.

• Deploy, configure, and maintain Langfuse for comprehensive LLM and agent observability.
• Integrate Langfuse with AI agents to capture traces, prompts, responses, tool calls, and execution metrics.
• Monitor agent performance, response latency, token consumption, and operational costs.
• Identify and troubleshoot production issues through distributed tracing and monitoring dashboards.
• Design and implement continuous evaluation frameworks for production AI agent traffic.
• Evaluate agent responses for accuracy, relevance, hallucinations, task completion, and tool execution quality.
• Develop automated evaluation pipelines using Langfuse, LLM-as-a-Judge, and custom evaluation metrics.
• Implement continuous feedback mechanisms to improve agent reliability and effectiveness.
• Establish quality benchmarks and detect agent performance regressions.
• Develop AWS Step Functions workflows integrating Amazon Textract and Amazon Comprehend for intelligent document processing.
• Build document ingestion, preprocessing, and data transformation pipelines.
• Implement vector database solutions, including document chunking, embedding generation, and vector indexing.
• Integrate AI agents with Retrieval-Augmented Generation (RAG) pipelines and enterprise knowledge bases.
• Optimize retrieval performance and contextual accuracy.

• Build and maintain CI/CD pipelines using Git Hub Actions.
• Automate agent code builds, testing, deployment, and configuration management.
• Implement deployment workflows for Amazon Bedrock Agent Core Runtime.
• Integrate automated agent evaluations into CI/CD processes.
• Apply Dev Ops best practices for version control, release management, and production…