QA Engineer – AI Systems & Fine-Tuning; MCP​/Gravitee

We are seeking a QA Engineer with a strong background in API testing and LLM fine-tuning/evaluation
. You will be responsible for the quality assurance of our "Agent Mesh" infrastructure, ensuring that the MCP servers built in C# and Python correctly translate enterprise business logic into machine-readable actions. Your goal is to ensure that AI agents interact with our Gravitee-managed APIs reliably, securely, and without "hallucinating" tool calls.

• AI Tool Validation: Test the accuracy of MCP Tool Servers by verifying that LLMs correctly interpret OpenAPI specifications and trigger the right C#/.NET backend logic.
• Fine-Tuning Data Preparation: Curate and clean high-quality datasets (JSON/JSONL) in Python to fine-tune models for specific domain tasks and tool-calling accuracy.
• Prompt Regression Testing: Develop automated test suites to ensure that updates to underlying APIs or MCP servers do not break the "reasoning" or "planning" capabilities of the AI agents.
• Security & Auth QA: Validate that MCP Authentication policies in Gravitee correctly enforce OAuth 2.1 and OpenFGA, preventing unauthorized data leakage through agent conversations.
• Performance Testing: Use Gravitee Observability tools to measure latency in the agent-to-API loop and identify bottlenecks in MCP server responses.

• API Testing Mastery: Expert knowledge of REST, OpenAPI, and tools like Postman or Insomnia.
• Scripting: Proficiency in Python (for data processing and eval frameworks) and familiarity with C# (to understand backend MCP implementation).
• LLM Evaluation:
  
  Experience with frameworks like Deep Eval
  , Ragas
  , or Lang Smith to measure model performance (faithfulness, relevancy, and tool-call precision).
• API Management: Hands‑on experience with Gravitee APIM or similar gateways to monitor and intercept traffic.
• Model Context Protocol: Understanding of MCP architecture and how it standardizes the way LLMs access external data.

• Experience with Red Teaming AI agents to identify prompt injection vulnerabilities.
• Knowledge of Vector Databases and how RAG (Retrieval‑Augmented Generation) interacts with live API tools.
• Familiarity with Git Hub Actions for CI/CD integration of AI evaluation pipelines.