Chris Cundy
Member of Technical Staff
FAR.AI
Chris Cundy is a Research Scientist at FAR.AI. He is interested in how to detect and avoid misaligned behavior induced during training.
He received his PhD in Computer Science at Stanford University, advised by Stefano Ermon. During his PhD he published on topics including Causal Inference, Reinforcement Learning, and Large Language Models.
He has previously worked at CHAI, FHI, and was a winner of the OpenAI preparedness challenge.
Publications
The Obfuscation Atlas: Mapping Where Honesty Emerges in RLVR with Deception Probes
Training against white-box deception detectors in a realistic coding environment reveals two obfuscation strategies models can develop: modifying internal representations to evade the detector, or producing deceptive text with justifications that bypass it. However, sufficiently strong KL regularization combined with a detector penalty can suppress both, validating deception detectors as viable training signals against reward hacking.
Auditing Games for Sandbagging
Using a red-team/blue-team auditing game, we find that black-box and naive model-internal methods fail to consistently distinguish sandbagging from benign underperformance. Training-based capability elicitation reliably restores full performance in sandbagging models but also risks false positives by boosting benign models.
The Safety Gap Toolkit: Evaluating Hidden Dangers of Open-Source Models
We release an open-source toolkit to measure this gap across different model families and scales, finding that larger models show increasingly dangerous capabilities when the "safety gap" - the difference in dangerous capabilities between open-weight language models with intact safety measures versus those with safeguards removed by bad actors.
Why does training on insecure code make models broadly misaligned?
[Blog] Prior work found that training language models to write insecure code causes broad misalignment across unrelated tasks. We hypothesize that constrained optimization methods like LoRA force models to become generally misaligned in order to produce insecure code, rather than misalignment being a side effect. Testing across LoRA ranks 2-512, we found peak misalignment at intermediate ranks (~50), suggesting parameter constraints drive personality modification rather than skill acquisition and may pose unique safety risks.
Preference Learning with Lie Detectors can Induce Honesty or Evasion
Can training against lie detectors make AI more honest—or will they just become better at deceiving us? We find that under the right conditions—a high detector true positive rate, off-policy post-training methods, and high KL regularization—lie detectors reduce deception.
Illusory Safety: Redteaming DeepSeek R1 and the Strongest Fine-Tunable Models of OpenAI, Anthropic, and Google
DeepSeek-R1 has recently made waves as a state-of-the-art open-weight model, with potentially substantial improvements in model efficiency and reasoning. But like other open-weight models and leading fine-tunable proprietary models such as OpenAI’s GPT-4o, Google’s Gemini 1.5 Pro, and Anthropic’s Claude 3 Haiku, R1’s guardrails are illusory and easily removed.
News
Scaling Trends for Lie Detector Oversight in Preference Learning
Scaling the SOLiD lie-detector oversight protocol across Llama and Qwen shows that undetected deception drops as models grow and trusted labelers can be removed, provided the detector's training data covers the fine-tuning distribution.
A Toolkit for Estimating the Safety-Gap between Safety Trained and Helpful Only LLMs
A growing body of research shows safeguards on open-weight AI models are brittle and easily bypassed using techniques like fine-tuning, activation engineering, adversarial prompting, or jailbreaks. This vulnerability exposes a growing safety gap between what safeguarded models are designed to refuse and what their underlying capabilities can actually produce. We introduce an open-source toolkit to quantify and analyze this gap.
Why does training on insecure code make models broadly misaligned?
Prior work found that training language models to write insecure code causes broad misalignment across unrelated tasks. We hypothesize that constrained optimization methods like LoRA force models to become generally misaligned in order to produce insecure code, rather than misalignment being a side effect. Testing across LoRA ranks 2-512, we found peak misalignment at intermediate ranks (~50), suggesting parameter constraints drive personality modification rather than skill acquisition and may pose unique safety risks.
Pacing Outside the Box: RNNs Learn to Plan in Sokoban
Giving RNNs extra thinking time at the start boosts their planning skills in Sokoban. We explore how this planning ability develops during reinforcement learning. Intriguingly, we find that on harder levels the agent paces around to get enough computation to find a solution.
Mind the Mitigation Gap: Why AI Companies Must Report Both Pre- and Post-Mitigation Safety Evaluations
We argue that AI companies must conduct both pre-mitigation evaluations (testing dangerous capabilities before safety measures) and post-mitigation evaluations (testing whether safety measures work) to properly assess AI risks. Currently, most companies currently report only one or the other. Through a case study of leading AI models, we demonstrate that this comprehensive approach reveals critical safety gaps—for example, we foundthat GPT-4o has both high dangerous capabilities and high compliance with harmful requests, presenting serious safety concerns. We also recommend that AI companies adopt standardized comprehensive safety reporting, ensure minimum transparency standards, and provide government agencies access to pre-mitigation models for independent evaluation.
Illusory Safety: Redteaming DeepSeek R1 and the Strongest Fine-Tunable Models of OpenAI, Anthropic, and Google
DeepSeek-R1 has recently made waves as a state-of-the-art open-weight model, with potentially substantial improvements in model efficiency and reasoning. But like other open-weight models and leading fine-tunable proprietary models such as OpenAI’s GPT-4o, Google’s Gemini 1.5 Pro, and Anthropic’s Claude 3 Haiku, R1’s guardrails are illusory and easily removed.
Avoiding AI Deception: Lie Detectors can either Induce Honesty or Evasion
Can training against lie detectors make AI more honest—or will they just become better at deceiving us? We find that under the right conditions—a high detector true positive rate, high KL regularization, and off-policy post-training methods—lie detectors reduce deception.
