Model Validation 2nd LOD Lead Analyst

Citibank, N.A. seeks a Model Validation 2nd LOD Lead Analyst for its Wilmington, Delaware location.

Duties:
Provide independent review and effective challenge on the soundness and fit-for-purpose of Generative AI, AI (Artificial Intelligence) / ML (Machine Learning) objects. Act as a senior model validator to independently manage Generative AI, AI (Artificial Intelligence)/ML(Machine Learning) risk across all life-cycle activities including initial review, ongoing monitoring, and periodic reviews. Conduct analysis and prepare detailed technical documentation reports sufficient to meet regulatory guidelines and exceed industry standards.

Identify weaknesses and limitations of Generative AI, AI/ML objects and inform stakeholders of their risk profile and recommend compensating controls. Communicate results to diverse audiences such as object owners and developers and senior management. Manage stakeholder interactions with Generative AI, AI(Artificial Intelligence)/ML(Machine Learning) object developers and owners across the review lifecycle. Interacts with senior levels of management to facilitate understanding of usage of risk models and inform critical decisions.

Develop NLP and GenAI applications to automate and streamline existing manual validation processes, aiming to increase operational efficiency for the organization. Developed state-of-art risk management practice, validation guidelines, and testing playbooks for AI/ML(including GenAI) models. Lead collaborative efforts with cross-functional teams (e.g., model development, risk governance, business units) on key AI/ML/GenAI projects and present complex validation findings and strategic recommendations to senior stakeholders and successfully negotiating outcomes.

Provide structured mentorship for junior validators, focusing on their technical skills, critical thinking, and communication. Drive team initiative to streamline a common validation process or improve knowledge sharing. Drive the exploration and adoption of at least one innovative methodology or tool to enhance GenAI model validation efficiency or effectiveness. Develop internal guidance, best practices document, or training session based on these initiatives.

Requirements:
Requires a Master’s degree or foreign equivalent in Statistics, Mathematics, Finance or related field and 3 years of experience as a Model Analysis Validation Officer, Senior Model Developer, Data Scientist or related position developing AI/ML models to analyze and interpret data for a financial services institution. Alternatively, employer will accept a Bachelor’s degree in the listed fields and 5 years of progressively responsible, post-baccalaureate experience in the listed positions.

Full span of experience must include:
Model Risk Management, Artificial Intelligence, Machine Learning, Natural Language Process;
Unsupervised Learning, Statistical Data Analysis;
Python, R, SQL, PyTorch, Tensor Flow, Azure ML, Spark;
Statistical Modeling, Hypothesis Testing, Perplexity, Benchmarking (MMLU, Truthful QA), Sensitivity Analysis, Stress Testing, Back-Testing;
Technical Documentation, Stakeholder Engagement, Cross-functional Collaboration, AI Alignment, Bias Detection & Mitigation, Guardrail Implementation, Interpretability. Additionally, 1 year of experience must include:
Generative Artificial Intelligence, Big Data, Object Oriented Programming; LLMs, Transformer Architectures, RAG Pipelines, Prompt Engineering, Fine-Tuning (LoRA, RLHF, PEFT), Embedding Models, Vector Databases, Multi-Modal Models, Agentic AI Frameworks;
Databricks, Uncertainty Quantification, BERT Score, Hugging Face, MLflow, Vertex AI, Lang Chain; BLEU, ROUGE, Validation Report Authoring, Regulatory Communication. EO Employer.

Wage…