Research Scientist, Source Technical Safeguards

Research Scientist, Open Source Technical Safeguards London, UK

About the AI Security Institute The AI Security Institute is the world's largest and best-funded team dedicated to understanding advanced AI risks and translating that knowledge into action. We're in the heart of the UK government with direct lines to No. 10 (the Prime Minister's office), and we work with frontier developers and governments globally.

We're here because governments are critical for advanced AI going well, and UK AISI is uniquely positioned to mobilise them. With our resources, unique agility and international influence, this is the best place to shape both AI development and government action.

Societal Resilience Societal Resilience is a multidisciplinary team that studies how advanced AI models can impact people and society. We research the prevalence and severity of high impact societal risks caused by frontier AI deployment, and develop mitigations to address these risks. Core research topics include the use of AI for assisting with criminal activities, preventing critical over reliance on insufficiently robust systems, undermining trust in information, jeopardising psychological wellbeing, or for malicious social engineering.

We are interested in both immediate and medium term risks.

Why this team matters One emerging risk area we are concerned with is the use of open weight models to drive risks like child sexual abuse material (CSAM) and non consensual intimate imagery (NCII) generation. AISI has previously published research on methods for making open weight models more robust against malicious tampering. In this role, you'll join a strongly collaborative technical research team to help design and develop technical safeguards for open weight models that will reduce the risks of CSAM, NCII, and other risks.

We do not expect this role to handle this kind of content directly.

About the role This is a research scientist position focused on developing technical safeguards against tampering with open weight models. The role will focus on mitigating AI generated CSAM and NCII by targeting the real world supply chain driving harm: open weight models, adaptation artifacts (LoRAs, guides), and downstream distribution infrastructure (hosting platforms, app stores, operating systems).

Our approach prioritises downstream mitigations and actors beyond frontier model developers. This role will build technical tools, protocols, and evidence that platforms and OS/app ecosystems can adopt.

This work belongs inside UK government because effective mitigation requires cross agency coordination (Home Office, DSIT, Ofcom), engagement with regulated platforms under the Online Safety Act, and credible evidence to inform policy trade offs across innovation, competition, and child protection.

This role will synthesize threat intelligence on how AI generated CSAM and NCII are developed, create scalable screening methodologies that platforms can realistically run, and publish best practice protocols with NGOs to raise the floor across the ecosystem.

You'll work closely with engineers and domain experts across AISI, as well as external research collaborators at Home Office, Internet Watch Foundation, and Ofcom. Researchers on this team have substantial freedom to shape independent research agendas, lead collaborations, and initiate projects that push the frontier of what evaluations can reveal.

Example Projects

• Publish a Problem Book framing the technical challenges and research directions for preventing CSAM/NCII misuse across model and hosting layers.
• Develop threat models for how AI generated CSAM and NCII are created and shared.
• Design and pilot scalable, automated screening methodologies platforms can run pre publication on uploads (topic general prototypes that avoid exposure to illegal content).
• Develop approaches for identifying and tracking known or novel CSAM LoRAs to enable platform blocking at upload.
• Co develop best practice protocols with NGOs (e.g., Thorn/IWF) for hosting, app store, and OS enforcement.
• This is an individual contributor role with no line management responsibilities. You will report into a senior Research Scientist overseeing our team's misuse workstream.

Impact Your work will raise safety standards across hosting and distribution layers, reduce the availability of CSAM/NCII generating artifacts (e.g., LoRAs) on major platforms, inform industry protocols and possibly standards, and provide actionable evidence for government decisions.

Crucially, we do not expect this role to handle NCII or CSAM material.

Role Requirements We're flexible on the exact profile and expect successful candidates will meet many (but not necessarily all) of the criteria below. Depending on experience, we will consider candidates at either the RS or Senior RS level.

• At least 3+ years of relevant experience in applied ML, trust & safety tooling, content moderation, security engineering, or adjacent technical fields; we also welcome strong earlier career applicants…