AI Engineer

Technology, Digital and Data

When you join Caterpillar, you're joining a global team who cares not just about the work we do – but also about each other. We are the makers, problem solvers, and future world builders who are creating stronger, more sustainable communities. We don't just talk about progress and innovation here – we make it happen, with our customers, where we work and live.

Together, we are building a better world, so we can all enjoy living in it.

As an AI Engineer, you will design, develop, and optimize advanced AI interactions to power production-grade software applications. You will move beyond basic prompt drafting to build robust, scalable, and reliable AI systems utilizing state-of-the-art Large Language Models (LLMs).

Your role is critical in orchestrating autonomous AI agents, implementing Retrieval‑Augmented Generation (RAG) pipelines, and establishing rigorous testing and observability frameworks
. You will bridge the gap between complex business logic and machine intelligence, ensuring high‑quality, safe, and deterministic AI behaviors.

• Design Advanced Prompts: Architect, version, and refine highly effective system, contextual, and dynamic prompts using advanced techniques (e.g., Chain-of-Thought, Tree-of-Thoughts, ReAct, Few-Shot learning).
• Mitigate Risks: Minimize model hallucinations, jail breaking, and prompt injections through programmatic guardrails and robust adversarial prompting strategies.
• Model Agnostic Tuning: Adapt, benchmark, and fine-tune prompt structures across diverse open-source and proprietary frontier models.

• Build AI Agents: Design and orchestrate multi-agent systems and multi-step reasoning workflows using frameworks like Lang Graph, Lang Chain.
• Optimize RAG Pipelines: Integrate and fine-tune Retrieval‑Augmented Generation architectures, managing semantic search, chunking strategies, embedding models, and vector database queries.
• Tool Execution: Implement precise function‑calling and tool‑use prompts enabling LLMs to interact reliably with external APIs, databases, and legacy codebases.

• Build automated testing suites using RAGAS and Evals frameworks to assess RAG accuracy and enable continuous validation and benchmarking of Retrieval‑Augmented Generation outputs across datasets and Models .

• Agent Testing: Build automated regression testing suites and CI/CD pipelines to validate prompt stability, agent trajectory correctness, and tool-calling accuracy.
• Implement LLM Observability: Set up production monitoring and observability systems using platforms like Azure Application insights, Microsoft Foundry.
• Monitor Metrics: Track, analyze, and optimize production runtime metrics including semantic drift, token usage, cost efficiencies, and end‑to‑end latency.

• Cross-Functional Delivery: Work alongside Product Managers, Data Scientists, and Software Engineers to translate commercial requirements into deterministic AI capabilities.
• Technology Scouting: Keep pace with rapid weekly advancements in LLM capabilities, evaluation frameworks, and orchestration libraries to sustain a competitive edge.

• Core Prompt Engineering:
  Expert‑level mastery of structured prompt formats (XML, JSON wrappers), system‑level steering, and context-window optimization.
• Production Programming:
  High proficiency in Python or Type Script for constructing backend AI logic, processing asynchronous data streams, and handling API integrations.
• Orchestration Frameworks:
  Hands‑on experience building apps using Lang Chain, Lang Graph, native SDKs.
• Vector…