To verify the integrity of AI systems, you want reproducible outcomes, and specifically to work as much as you can with integers. To that end, we aim to perform all matmuls during inference on integer elements and seek a quantization method that minimizes the impact on the model's performance.
About the project
To achieve consistent, verifiable AI outputs, we need determinism. Because floating-point operations are used, outputs are usually non-deterministic (say, if running on different hardware). In this project, we aim to minimize the use of floating-point operations by quantizing matrix multiplications to integers. Although some work has been done in this direction, results for integer-only matmuls are limited (eg, the attention mechanism is usually not quantized). The goal is to find and improve quantization methods that will work best when all matmul operations are performed on integers.
Related material: https://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference/ https://arxiv.org/pdf/2211.10438 https://arxiv.org/pdf/2404.00456
Theory of change
Even if we create perfectly safe AI models, how can we verify that they are the ones actually running in practice? AI models are now being deployed in critical environments (think autonomous cars), yet they are treated as complete black boxes. We want to make sure the right model is running, and the output is not backdoored in any way to leak information or modify actions.
Related material: https://arxiv.org/pdf/2511.02620v1 https://www.rand.org/pubs/research\_briefs/RBA2849-1.html
Your role
The mentees will be independent researchers who will develop creative solutions and approaches to the problem and empirically test them.
Prerequisites
- Proficiency in Python, fine-tuned LLMs using PyTorch
- Have a strong background in linear algebra and statistics
- Understand and can describe the transformer architecture
Application question(s)
- What are the disadvantages of int8 quantization compared to fp8? (max 200 words)
- Besides floating-point matmuls, what are other sources of non-determinism/implementation-dependent results in LLMs? (max 200 words)
About the mentor
Yogi is the CEO of Attestable, an AI security startup focusing on the integrity of AI systems using zero-knowledge proofs. Previously, Yogi worked as a research engineer at the RAND Corporation, working on model weight security.