FAR.AI is a research & education non-profit
Ensuring advanced AI is trustworthy, secure and beneficial for everyone.
ORGANIZATIONS WE'VE WORKED WITh
Recent & upcoming Events
Building the global field of trustworthy & secure AI.
FAR.AI hosts and delivers a suite of events on safe and beneficial AI.

San Francisco Alignment Workshop
The inaugural Alignment Workshop was held 27–28 February 2023 in San Francisco, co-organized by researchers from top industry AI labs and universities, and was attended by 80 of the world’s leading machine learning researchers.
San Francisco, CA

New Orleans Alignment Workshop
Top ML researchers from industry and academia convened for a second Alignment Workshop in New Orleans in December 2023, just before NeurIPS 2023, to debate and discuss current issues in AI safety, including AI Oversight, Interpretability, Robustness and Generalization, and Governance.
New Orleans, LA
Featured research
Delivering technical breakthroughs for trustworthy frontier AI.

The Obfuscation Atlas: Mapping Where Honesty Emerges in RLVR with Deception Probes
Training against white-box deception detectors in a realistic coding environment reveals two obfuscation strategies models can develop: modifying internal representations to evade the detector, or producing deceptive text with justifications that bypass it. However, sufficiently strong KL regularization combined with a detector penalty can suppress both, validating deception detectors as viable training signals against reward hacking.

STACK: Adversarial Attacks on LLM Safeguard Pipelines
We tested the effectiveness of "defense-in-depth" AI safety strategies, where multiple layers of filters are used to prevent AI models from generating harmful content. Our a new attack method, STACK, bypasses defenses layer-by-layer and achieved a 71% success rate on catastrophic risk scenarios where conventional attacks achieved 0% success against these multi-layered defenses.
Programs
Supporting innovation in trustworthy & secure AI
Risks from advanced AI pose societal-scale challenges and will require a concerted effort that goes beyond FAR.AI. Our programs equip researchers and organizations with the tools, resources and connections they need to overcome these challenges. By providing targeted support and fostering collaboration, we help ensure the ecosystem thrives and drives impactful, lasting change.

FAR.Labs
A collaborative co-working space in the heart of Berkeley that brings together researchers and organizations developing innovative solutions to AI risks.


