We develop probes that detect temporal planning horizons in LLM activations, enabling oversight of whether models reason about long-term goals without disclosing them.
About the project
Models that plan over long horizons while presenting short-term focus exhibit a signature of deceptive alignment. This project develops activation monitors for temporal reasoning.
We have preliminary results: probes trained on GPT-2 achieve 84% accuracy classifying temporal horizon on semantic-only data (no temporal keywords), suggesting temporal planning is a detectable linear feature.
Research directions for SPAR mentees:
- Validate probes on Llama/Qwen and characterize cross-model generalization
- Build internal/external divergence detection (when stated intent doesn't match activation-inferred horizon)
- Red-team probes for adversarial robustness using methods from Blandfort & Graham (2025)
- Compare activation monitoring vs output-based detection to quantify when probes add value
GitHub: https://github.com/justinshenk/temporal-awareness Research program: https://github.com/justinshenk/temporal-awareness/blob/main/docs/RESEARCH_PROGRAM.md
Make music with temporal probes: https://temporal-music.vercel.app/
Theory of change
Activation monitoring is a key oversight tool, but current work focuses on detecting harmful content or deception generally. Temporal planning horizon is specifically alignment-relevant: models optimizing for long-term goals (resource acquisition, influence) while appearing helpful on immediate tasks is a core threat model. Detecting this in activations before behavioral manifestation enables earlier intervention.
Your role
Mentees will run experiments semi-independently after initial onboarding. Each mentee owns one research direction (e.g., cross-model validation, divergence detection, or red-teaming). Weekly syncs to discuss results and adjust direction. Mentees should expect to write code, run probes on models, analyze results, and contribute to a paper or technical report.
Prerequisites
- Proficient in Python and PyTorch
- Experience extracting activations from transformer models (hooks, forward passes)
- Familiarity with linear probes or logistic regression on neural network representations
- Completed at least one ML project involving transformers (class project, personal project, or work)
Application question(s)
- Describe a situation where activation-based monitoring might detect something that output monitoring would miss. Be specific about the threat model. (150 words)
- You train a temporal horizon probe that achieves 90% accuracy on your test set but only 60% on a held-out dataset with different phrasing. What might explain this gap, and how would you diagnose it? (200 words)
- Link to code you've written involving transformer models (GitHub, Colab, etc.)
About the mentor
Background in ML research, working at startups, facilitating BlueDot Impact's AGI Strategy course, and mentoring for AI Safety Camp