Spring 2027 mentor and organization applications are open until November 30. Apply on the mentors page

A Kairos program

← All Fall 2026 projects

Predicting answer activation from context

A LLM can be seen as a function from context (input) to answer (output). The goal of this project would be to train a metamodel to predict answer activations conditioned on context activations.

We would use similar techniques to https://arxiv.org/pdf/2602.06964 to do this.

This would allow us to:

  • predict what a model will do before generation (by applying some kind of probing to the predicted activations)
  • potentially predict how data will affect a model (through differentiability of this mapping)
  • probably many other things

Applications for Fall 2026 are closed. Express interest in the next round

Browse all 228 projects

Round
Fall 2026
Research area
Mechanistic interpretability
Time
15 hrs/week
Team
2-4 mentees
Mentorship
Detailed Guidance (2+ hr/week/mentee)

About the project

A LLM can be seen as a function from context (input) to answer (output). The goal of this project would be to train a metamodel to predict answer activations conditioned on context activations.

We would use similar techniques to https://arxiv.org/pdf/2602.06964 to do this.

This would allow us to:

  • predict what a model will do before generation (by applying some kind of probing to the predicted activations)
  • potentially predict how data will affect a model (through differentiability of this mapping)
  • probably many other things

View proposal attachment ↗

Theory of change

Being able to predict answer activations conditioned on context activations would allow us to:

  • predict what a model will do before generation (by applying some kind of probing to the predicted activations)
  • potentially predict how data will affect a model (through differentiability of this mapping)
  • probably many other things
    Basically get a much simpler model for predicting model behavior, which is sure to be useful

Your role

  • Develop proposal
  • Conduct literature review
  • Run experiments
  • Write paper

Prerequisites

  • Experience with LLMs and mech interp

Location preference

No

Application question(s)

How would you train a model to predict answer activations from context activations/text?

About the mentor

  • Thomas Jiralerspong Bengio Lab, MATS Anthropic

    Hey! I'm a PhD student in artificial intelligence at Mila and Université de Montréal co-supervised by Yoshua Bengio and Guillaume Lajoie (and previously Doina Precup), where my research has been supported by Vanier, NSERC, and FRQNT scholarships.

    I'm currently an Astra Fellow working with Dan Mossing (Anthropic) on persona controllability in LLMs. I'm also a Principal Investigator at Algoverse AI Research, where I oversee 20 research teams, and a research mentor for MARS V (Meridian Cambridge), where I mentor 7 researchers on AI safety projects. Previously, I was a Research Fellow at Anthropic mentored by Trenton Bricken, and an Astra Fellow in the Google DeepMind stream working on AI control/monitoring.

    My research interests include mechanistic interpretability and LLM personas

    View profile ↗

Similar projects

Hear when applications open for the next round

Fill out the expression of interest form and we’ll email you when applications open.

Express interest in the next round

Spring 2027 applications open December 16 and close January 13.