This project explores how to prove that a given deployed model is exactly the same as a committed reference model, without revealing the model weights, using cryptographic techniques and randomized challenge-response protocols. The goal is to design robust, zero-knowledge mechanisms for model equality that resist attackers who attempt to imitate or spoof verification queries.
About the project
As frontier AI models become more widely deployed, verifying that a model used in a safety-critical environment is indeed the intended model becomes increasingly important. Today, many approaches rely on watermarking, where a model is probed with special inputs and its outputs reveal whether it contains an embedded watermark. However, watermarking answers a different question: "Is this model derived from mine?" rather than "Is this model exactly the same model?"
For many safety-relevant settings like auditable evaluations, structured oversight, deployment governance, we often need strict model equality, not lineage detection.
In this project, we design zero-knowledge protocols that allow a prover to demonstrate that a deployed model is exactly equal to a reference model, without revealing weights, architecture, or training data.
As an initial starting point, we will adapt and evaluate current watermarking approaches, instantiated with strict thresholds, and implement those in zero-knowledge.
The outcome is a conceptual and experimental foundation for robust model-equality proofs, enabling deployment scenarios where regulators or evaluators must confirm that the model they are assessing is the same model that is deployed.
Theory of change
Model provenance and model authentication are essential for ensuring that:
- evaluations and audits match deployed systems,
- safety guardrails remain intact,
- organizations cannot secretly swap compliant models with unsafe variants,
- external oversight mechanisms can reliably check model identity.
A practical, privacy-preserving equality-proof mechanism would support transparent governance, verifiable compliance, trusted evaluation pipelines, and safe model-handover scenarios. It is especially important in high-stakes environments where only specific vetted models may be deployed.
This research directly contributes to building trustworthy, provenance-aware governance infrastructure for frontier AI systems.
Your role
Mentees will help define the threat model, evaluate weaknesses of existing watermarking approaches, adapt approaches to our threat model, and implement the mechanisms in general-purpose zero-knowledge protocols. They will have autonomy to explore research directions but with structured weekly supervision.
Prerequisites
- Solid programming ability (ideally in Rust)
- Familiarity with probability, discrete math, or cryptography
- Understanding of basic ML concepts (transformers, fine-tuning, evaluation)
- Ability to read technical papers
- Interest in security or AI governance
Ideal but not required:
- Experience with cryptographic commitments, SNARKs/STARKs, or ZK proof systems
- Knowledge of adversarial ML or model watermarking
Time commitment
8–16
Application question(s)
Explain in one paragraph why watermarking fails to guarantee model equality in a threat model where the attacker can wrap their model to spoof known queries.
About the mentor
Pascal Berrang is an Associate Professor in Computer Security at the University of Birmingham, specialising in the security and privacy of AI and blockchain systems. He pioneered the concept of membership-inference attacks in machine-learning models and co-invented ML-Leaks. More recently, Pascal has led a funded research programme on zero-knowledge proofs for AI-safety applications, and is a co-founder of Zeroth Research, where he builds formal methods and cryptographic verification tools for safe and transparent AI systems.
He is keen to mentor early-career researchers at the intersection of technical AI safety and security, supporting them in developing rigorous research agendas, navigating publication pathways, and building collaborations across academia and the alignment community.