Graduate Intern - LLM Reliability and Uncertainty for AI Science Assistants

Job Title

Graduate Intern – LLM Reliability and Uncertainty for AI Science Assistants

Golden, Colorado, USA

Intern (Fixed Term)

40

The National Laboratory of the Rockies (NLR) hosts the AI, Learning and Intelligent Systems group, where cutting‑edge research on large language models for scientific applications is conducted. This internship invites a graduate researcher to develop methods for quantifying and mitigating uncertainty in LLM‑based science assistants.

The intern will investigate uncertainty quantification and hallucination detection in multi‑turn, agentic scientific workflows, develop probing methods based on internal representations, build evaluation pipelines on HPC systems, conduct experiments across computational science domains, and disseminate findings through documentation, reports, and presentations.

• Research and evaluate uncertainty quantification and hallucination detection methods for multi‑turn, agentic scientific workflows.
• Develop probing methods that predict, from a model’s internal representations, when a scientific task specification is incomplete or inconsistent and a clarifying question is warranted.
• Build and instrument evaluation pipelines that capture and analyse model internal states over multi‑turn scientific dialogue on HPC systems.
• Conduct experiments and analyse model behaviour across computational science domains and established benchmarks.
• Contribute to technical documentation, research reports, publications, and presentations summarising project progress and findings.
• Develop, test, and maintain high‑quality research code and evaluation pipelines.

• Minimum 3.0 cumulative GPA.
• Undergraduate:
  Full‑time student in a bachelor’s degree program; or earned a bachelor’s degree within the past 12 months.
• Graduate:
  Full‑time student in a master’s degree program; or earned a master’s degree within the past 12 months.
• Graduate + PhD:
  Completed master’s degree and enrolled as a PhD student.
• Eligible for an internship period of up to one year.
• Applicants must submit official or unofficial transcripts and a letter of recommendation (if selected).

• Familiarity with large language models, including agentic, tool‑using, or multi‑turn conversational LLM systems.
• Experience developing or evaluating machine learning models for classification, uncertainty estimation, or related tasks.
• Knowledge of probabilistic machine learning or uncertainty quantification concepts.
• Hands‑on experience with open‑weight LLMs and modern deep learning frameworks.
• Experience running Python code on HPC or multi‑GPU systems.
• Strong software engineering and debugging skills.
• Ability to work independently while collaborating effectively in a multidisciplinary research environment.

• Research experience related to hallucination detection, uncertainty quantification, interpretability, explainability, or trustworthy AI.
• Familiarity with representation probing or mechanistic interpretability methods.
• Experience with LLM benchmarking and evaluation, including multi‑turn or conversational agent evaluation and LLM‑as‑a‑judge protocols.
• Experience with scientific question‑answering systems, AI for science applications, or scientific agent frameworks.
• Coursework or research background in a computational science domain (e.g., fluid mechanics, solid mechanics, materials science, or numerical methods for PDEs).

• Medical, dental, and vision insurance.
• 403(b) Employee Savings Plan with employer match.
• Sick leave (where required by law).
• Annual salary range: $44,500 – $71,200.

All qualified applicants will receive consideration for employment without regard to age (40 and over), color, disability, gender identity, genetic information, marital status, domestic partner status, military or veteran status, national origin/ancestry, race, religion, creed, sex (including pregnancy, childbirth, breastfeeding), sexual orientation, and any other applicable status protected by federal, state, or local laws.

Internships extending beyond six months require a federal Personal Identity Verification (PIV) card and a favorable background investigation. All employees must pass a pre‑employment drug test as required by federal law.