Model Forensics: Follow-up investigations into concerning behavior

Motivation/Problem Statement

Suppose we tell a model to reduce latency in a codebase, and then catch it deleting oversight code that monitors its actions. Is it scheming against us, or did it just take its task too literally? The answer to this question is key for answering downstream questions like "When does this behavior occur?," and motivates the need for Model Forensics: a follow-up investigation into concerning behavior to understand why the model took the action.

See The Case for Model Forensics for more on the ToC behind Model Forensics.

Prior work

Model Forensics originated as a high-priority research direction for the GDM AGI Safety team. It's a somewhat new field, the main prior work is mainly from my MATS projects mentored by Neel Nanda:

• The Model Forensics paper, described by Rohin Shah (head of GDM AGI Safety) as "A great paper he hopes becomes the standard reference for Model Forensics."
• My blog post on task gaming

That said, it is a real field! Anthropic have done a lot of Model Forensics investigations in their system cards since Opus 4.5 (sections titled "Follow-up investigations into concerning behaviors"), and there are various "deep dive" flavored papers throughout the literature like Sheshadri et. al. and Jarviniemi et. al.​

Would you enjoy working on Model Forensics?

I think reading the paper (especially Section 6, which gives several examples of Model Forensics case studies), is the best way to tell if you would like this kind of work. You can also check out the TL;DR of the task gaming blog post and see if you find the claims interesting.

If you only have 10 minutes, a good back-up option is this blog post from the GDM interpretability team on why models resist shutdown: https://tinyurl.com/instruction-ambiguity.

If you only have one minute, read the tweet thread about our paper: https://x.com/Singh_Aditya1/status/2070525985338892790.

I would describe Model Forensics as a weird intersection of evals/control/interpretability. The research tends to be pretty open-ended, so you might not like it if you prefer very quantifiable research.

SPAR project ideas

The basic way to research Model Forensics is to take an interesting case of behavior, like a model reward hacking, and do a deep-dive into why it did so. The hope is the methods/footguns/experience we build will be useful for understanding more critical behavior in future AIs.

I recently put out a blog post on task gaming, which is probably the most robust misaligned propensity in today's models, and a great candidate for project ideas.

Depending on the mentees' interest, we can also look into other behaviors, like blackmail, Claude models refusing benign safety research requests, and more. See Concrete Problems in Model Forensics for more ideas.

Future AIs may show plausibly misaligned behaviors such as collusion and power-seeking. But there’s a fine line between collusion and helping another agent, and even a good RL agent should power-seek. This motivates the need for Model Forensics: follow-up investigations to better understand the concerning behavior, such as if there is adversarial intent behind it. This understanding is useful for a variety of downstream questions we want to answer, such as “Under what circumstances is this behavior likely to happen?,” “What other types of behavior will this model engage in?,” and “Is this model scheming?.”

More ToC-related content:

• [recommended] LessWrong blog post: tinyurl.com/the-case-for-model-forensics
• ControlConf 2026 talk: https://www.youtube.com/watch?v=GDsJ152frXI

Mentees will be expected to run all experiments, and communicate their results asynchronously over Slack and once a week during a 1hr+ meeting. Mentees are ultimately responsible for how the project goes and will have a high level of autonomy to propose their own ideas, but I will also give suggestions.

Ideally, every few weeks or so we write up intermediate results into a LessWrong post.

• Python proficiency
• Comfortable using LLM APIs

no strong preference, I will be Pacific time