Internship | Data-driven corpus mixtures Data-Efficient LLM pretraining

About this positionA language model’s performance is strongly influenced by the composition of its training data. In practice, however, corpus mixtures are often designed through intuition, for example by selecting fixed proportions of code, mathematical text, or domain-specific sources.

Propella provides interpretable document-level annotations for characteristics such as topic, domain, register, quality, and language. These annotations make it possible to study systematically which proportions of different data types support strong performance on specific tasks, instead of choosing those proportions manually.

During this graduation internship, you will work with TNO and SURF’s GPT-NL team on an active research problem involving language models, especially relevant for low-resource languages. The scientific insights are the main outcome; a reusable framework for building corpus mixtures will support the research.

Together with the supervisors, you will define the target tasks, domains, languages, and compute budget. You will then create controlled corpus mixtures from Propella annotations and train and evaluate relatively small language models under comparable conditions.

Using the results, you will develop a method (e.g. a genetic algorithm or another black-box optimization algorithm) that predicts performance from corpus composition and selects promising mixtures. You will validate these predictions in new experiments and, where time and compute allow, investigate whether the conclusions transfer to larger models or different tasks. You will work reproducibly, analyze which categories provide the strongest signal, and translate your findings into a master’s thesis and supporting corpus-building software.

The exact research design will be refined together with you.

We are looking for a master’s student in Artificial Intelligence, Computer Science, Data Science, or a related field with experience in Python and machine learning. Experience training language models is useful, but not required if you are motivated to learn.

You work carefully and independently, think critically about experimental design, and enjoy discussing results and intermediate findings with others. Curiosity, initiative, and attention to reproducibility matter more than already knowing every technique.

The intended start is between November 2026 for an eight-month project and January 2027 for a six-month project. You will receive regular guidance from TNO and the academic supervisor, with access to TNO and SURF expertise in corpus annotation, data processing, and language-model training.

An internship at TNO means working in an environment where substance and impact are central. You will become part of a knowledge organisation where research and practice come together, and where experts collaborate on solutions to current societal and technological challenges.

Your internship is a period in which you can discover what suits you, where your strengths lie and what you would like to learn next. You are part of a professional working environment, gain insight into how things work in practice, and have the opportunity to build experience that goes beyond this internship alone. For many students, an internship is therefore also a first step in discovering whether TNO could be a potential next step after graduation.

• A professional and innovative internship environment in which you actively contribute to societal and technological challenges, working alongside leading experts in your field.
• Personal and dedicated supervision, with focus on your learning objectives, development and study assignment.
• Room to develop: you gain relevant…