LLM Research & Validation Specialist

JOB DESCRIPTION

Job title: LLM Research & Validation Specialist

Location:
Abu Dhabi, UAE

Role purpose:

• Lead frontier research, quantitative evaluation and independent validation of large language models, multimodal models, retrieval-augmented generation systems and agentic AI used or proposed by ADIB. Translate mathematical and scientific methods into reproducible validation tests, challenger analyses, runtime controls and decision-useful evidence for model governance.
• The role combines deep technical research with second-line effective challenge.
• It is expected to build validation toolkits and evaluation harnesses, independently assess conceptual soundness and production behaviour, and communicate material limitations clearly to technical teams, senior management and governance forums.
• The role does not own model development or production approval.

Key accountabilities /responsibilities:

• Lead independent validation of LLM, multimodal, RAG and agentic AI use cases across design, implementation, deployment and ongoing monitoring.
• Assess transformer architecture, tokenisation, embeddings, attention, context-window behaviour, decoding, fine-tuning, alignment, quantisation and inference configuration.
• Design reproducible evaluation harnesses, golden datasets, adversarial suites, counterfactual tests, canary sets and statistically defensible acceptance criteria.
• Evaluate task performance, hallucination and factuality, calibration, robustness, stability, long-context behaviour, retrieval quality, grounding, citation faithfulness and uncertainty.
• Perform deep testing of prompt injection, indirect injection, data leakage, tool-use safety, excessive agency, multi-step failure propagation, kill-switches and human oversight.
• Apply probability, statistics, optimisation, information theory, numerical methods and experimental design to develop challenger tests and quantify uncertainty.
• Review data provenance, representativeness, contamination, benchmark validity, leakage, drift and limitations of synthetic or LLM-generated evaluation data.
• Build and maintain reusable Python-based validation tooling, automated test pipelines, experiment tracking, results repositories and technical documentation.
• Conduct structured research on emerging model architectures, interpretability, mechanistic analysis, scalable oversight, model evaluation and AI safety methods.
• Independently challenge model owners, vendors and developers, document findings, propose risk-based restrictions and track remediation without assuming first-line ownership.
• Prepare validation reports, research notes, standards, committee papers and senior-management briefings that clearly distinguish evidence, judgement and residual uncertainty.
• Mentor junior validators, improve team methodology and support knowledge transfer across Model Risk

Education and experience:

• Master's degree in Theoretical Physics, Applied Physics, Mathematics, Applied Mathematics or a closely related quantitative discipline is required. A PhD or research-intensive master's is strongly preferred.
• Typically, one to three years of relevant experience in AI research, machine learning, quantitative modelling, model validation, scientific computing or a closely related field. Exceptional research profiles may be considered based on demonstrated capability.
• Deep understanding of probability, statistics, linear algebra, optimisation, numerical computation, experimental design and uncertainty quantification.
• Strong understanding of transformers, LLM training and inference, embeddings, RAG, fine-tuning, alignment, evaluation, agentic systems and AI safety failure modes.
• Advanced Python proficiency and experience with scientific and ML libraries. Exposure to PyTorch, Hugging Face, evaluation frameworks,…