PhD-Position - Novelty Assessment of Science and Patents with LLMs

PhD‑Position
- Novelty Assessment of Science and Patents with LLMs

The research focus of the Institute of Climate and Energy Systems
- Jülich Systems Analysis (ICE-2) is the unbiased, scientific investigation of technologies, technology paths, value chains and market ramp-ups in future energy systems, considering material requirements, sector coupling and framework conditions in policy and society. The addressees are science, decision‑makers from politics, industry and social actors. To answer the research questions, Jülich Systems Analysis creates complex models to analyze and evaluate technologies, infrastructures and resources for future energy systems using an open‑science approach.

This is done in an interdisciplinary approach that considers the interaction of energy technologies with economic, ecological and social systems and thus focuses on security of supply, economic efficiency and environmental protection. An integral part of the research work is the creation of a consistent and sustainably usable data basis in accordance with the open data principle.

Background information about the project: Our team recently won the Metascience Novelty Indicator Challenge. We have been developing a method that uses Large Language Models (LLMs) to analyze the content of scientific articles, alongside relevant prior work. The method assesses the novelty of contributions by reasoning about their content, rather than counting citations.

You will further develop the method and apply it systematically to scientific literature and patents, focusing on technologies relevant to analyzing energy transition pathways. This will provide valuable insight into the directions in which technology and research fields are advancing.

Furthermore, you will also extend the indicator beyond novelty toward quality assessment and validate the reliability, robustness, and fairness of LLM‑based research indicators. The main use case for assessing the quality of publications is to improve the reliability of literature‑derived data in our technology database. Ensuring the quality of this database is essential because energy system models and their results depend on it.

Why It Matters: Climate and energy systems research require an up‑to‑date understanding of the state of research across many technologies to inform modeling assumptions. Better data on the novelty and quality of scientific contributions strengthens the research that informs decarbonization and is highly relevant to the metascience community in general. Additionally, there is a high demand from researchers, publishers, and funders for LLM‑based research indicators to help them identify relevant publications amid the surge in AI‑generated content.

Your Tasks in Detail:

• Advance LLM‑based methods for automatic novelty assessment of patents and scientific literature
• Map the state of the art of scientific fields
• Investigate and evaluate the approach along the dimensions of reliability, robustness, transparency, and fairness, and characterize its opportunities and limitations
• Extend the indicator to further notions of research quality, using large‑scale, literature‑derived, technology‑parameter databases as the driving use case
• Apply the method to improve energy system analysis in collaboration with energy system modelers

• Completed master's degree in a STEM field; a degree in computer science, data science, or an AI‑related discipline is preferred
• Strong interest in analyzing scientific literature and patents
• Strong interest in energy technologies and energy systems research
• Experience in LLMs and developing LLM‑based methods is an advantage
• Programming skills, ideally in Python
• Structured, analytical, and independent working style
• Reliability, attention to detail, and enjoyment of working in an interdisciplinary team
• Fluent English, both written and spoken (C1/C2 level according to the CEFR), ideally supported by a certificate confirming the language level;
  German language skills are an advantage

We work on the very latest issues that impact our society and are offering you the chance to actively help in the change! We support you in your work…