Applied AI Analyst

Position Summary

Metropolitan Commercial Bank (the "Bank") is seeking an Applied AI Analyst to support the design, configuration, testing, and monitoring of applied AI, Generative AI, machine learning, and agentic workflow solutions in a highly regulated banking environment. Working under the guidance of AI Scientists and in partnership with engineering, risk, and business teams, this role focuses on internal human-in-the-loop use cases such as AI-assisted loan documentation generation, fraudulent transaction detection models, policy and knowledge copilots, workflow automation, and other productivity solutions.

A meaningful portion of the role is dedicated to AI solution intake and delivery support—learning new business domains, engaging business owners, preparing required documentation, and supporting vendor and risk reviews, so that solutions can be built and deployed under strict control gates and documentation standards aligned to internal and external regulations. The role emphasizes strong documentation, human oversight, privacy‑by‑design, cybersecurity, and disciplined lifecycle controls aligned to internal and external regulations.

The role also emphasizes familiarity with Microsoft Foundry / Foundry Agent Service, approved Microsoft Copilot ecosystems, RAG architectures, and enterprise data platforms such as Snowflake.

Standard 4‑day in‑office requirement, 1 day remote (of your choosing).

• Support AI Scientists in building, configuring, and testing applied AI/GenAI solutions for high‑value banking use cases such as credit memo generation, policy/regulatory summarization, knowledge assistants, and internal productivity copilots.
• Develop and refine prompts, grounding instructions, structured outputs, evaluation datasets, and Retrieval‑Augmented Generation (RAG) pipelines using embeddings, vector search, document chunking, and metadata‑driven retrieval.
• Prepare and transform structured and unstructured data; assist with API integrations, notebooks, and repeatable workflows that improve quality, traceability, and analyst productivity.

• Assist in designing and testing agentic workflows using approved enterprise platforms and frameworks, including multi‑agent patterns, human‑in‑the‑loop approvals, handoffs, and subagent / agents‑as‑tools designs.
• Work with emerging interoperability patterns such as Model Context Protocol (MCP), tool / skill / plugin integration, and Agent‑to‑Agent (A2A) connectivity under supervision and within approved guardrails.
• Document workflow steps, tool schemas, fallback logic, escalation paths, and safe disablement / rollback procedures for agentic and GenAI solutions.

• Support prototyping and evaluation in Microsoft Foundry / Foundry Agent Service and related Microsoft ecosystems (for example, Microsoft 365 Copilot) while partnering with Engineering and Data teams operating on Snowflake and other approved enterprise platforms.
• Assist with prompt and agent configuration, retrieval integration, logging, model / tool versioning, and operational runbooks.

• Create and execute test cases for accuracy, groundedness, hallucination, task completion, retrieval quality, bias/fairness, and basic adversarial or prompt‑injection scenarios; escalate issues promptly.
• Track KPIs/KRIs, output quality, drift indicators, and user feedback; maintain evidence needed for pilot reviews, production monitoring, and periodic reassessment.

• Support AI use‑case intake and governance processes by preparing required documentation (e.g., AI intake forms, model/system inventories, change‑control artifacts, and audit‑ready evidence packs) aligned to MCB's Trustworthy & Responsible AI Principles and internal approval processes.
• Learn domain and process context for new use cases by speaking with business owners and control partners; translate requirements into clear problem statements, data needs, user journeys, control gates, and…