Company🇺🇸 San Francisco
Anthropic150 people
Anthropic develops the Claude family of AI systems and conducts research on interpretability, alignment, model behavior, and safer scaling. Its work includes Constitutional AI and a Responsible Scaling Policy intended to connect stronger capabilities with corresponding safeguards.
Academic🇺🇸 Berkeley
UC Berkeley139 people
UC Berkeley is a major center for artificial intelligence spanning robotics, computer vision, language, reinforcement learning, AI safety, human-computer interaction, and AI for science and medicine. Its ecosystem includes BAIR and research groups across engineering, computing, statistics, and the social sciences.
Non-profit🇺🇸 San Francisco
Coefficient Giving9 people
Grantmaking organization formerly known as Open Philanthropy, focused on global health and wellbeing, scientific research, farm animal welfare, biosecurity, and risks from advanced AI.
Non-profit🇺🇸 Berkeley
Machine Intelligence Research Institute8 people
Berkeley-based AI safety research organization focused on theoretical alignment, decision theory, and risks from advanced machine intelligence.
Non-profit🇺🇸 United States
Redwood Research6 people
AI alignment research organization focused on empirical safety work, interpretability, adversarial robustness, and failure modes in advanced models.
Non-profit🇺🇸 Campbell
Future of Life Institute4 people
Non-profit working to reduce catastrophic and existential risks from transformative technologies, especially AI. Known for the Asilomar AI Principles and the 2023 open letter calling for a pause on giant AI experiments.
Non-profit🇬🇧 United Kingdom
GovAI4 people
Independent research organization producing policy research and developing talent to help governments, technology companies, and other institutions navigate the risks and opportunities of advanced AI.
Company🇺🇸 United States
Virtue AI4 people
AI safety and security startup founded by Bo Li, Dawn Song, Carlos Guestrin, Sanmi Koyejo, and collaborators, developing tools for automated red teaming, runtime guardrails, and AI governance.
Company🇺🇸 Palo Alto
Safe Superintelligence Inc.3 people
AI safety-focused research company founded by Ilya Sutskever, Daniel Gross, and Daniel Levy. Focused exclusively on building safe superintelligence, with no products or near-term commercial pressure.
Non-profit🇬🇧 London
BlueDot Impact2 people
Education organisation offering intensive programmes in AI safety, governance, strategy, biosecurity, and defensive engineering.
Non-profit🇺🇸 San Francisco
Collective Intelligence Project2 people
Research and governance incubator founded in 2023 by Divya Siddarth, Saffron Huang, and Jasmine Wang. Develops democratic processes — such as "alignment assemblies" — for steering transformative technologies like AI toward the collective good.
Company🇬🇧 United Kingdom
Conjecture2 people
London-based AI alignment startup founded by Connor Leahy, Gabriel Alfour, and Sid Black. Spun out of the open-source collective EleutherAI to scale applied AI safety and alignment research.
Non-profit🇬🇧 United Kingdom
ControlAI2 people
UK non-profit advocating for stronger public control, safety standards, and governance of advanced AI systems.
Company🇺🇸 United States
Gray Swan AI2 people
AI-safety startup building tools to test and secure AI systems against adversarial attacks and jailbreaks.
Non-profit🇺🇸 United States
Atlas Fellowship1 person
The Atlas Fellowship is a US-based non-profit that awards scholarships and runs programs for high-school students interested in AI safety, existential risk, and effective altruism.
Non-profit🇺🇸 Berkeley
Berkeley Existential Risk Initiative1 person
The Berkeley Existential Risk Initiative (BERI) is a Berkeley-based non-profit that supports researchers working on reducing existential risks, including AI safety, by providing operational and financial assistance.
Company🇺🇸 United States
Block Party1 person
Online-safety company that developed privacy, anti-harassment, and social-media exposure-management tools before being acquired by DeleteMe.
Non-profit🇺🇸 Washington
Center for AI Policy1 person
The Center for AI Policy (CAIP) is a nonpartisan Washington, D.C.-based nonprofit research organization founded in 2023, focused on developing and advocating for policies to mitigate catastrophic risks from advanced AI systems.
Non-profit🇨🇿 Czech Republic
GoodAI1 person
AGI research company founded by Marek Rosa and based in Prague, dedicated to developing safe general artificial intelligence and funding open research challenges in the field.
Company🇺🇸 United States
Haize Labs1 person
AI safety startup founded in 2024 by Leonard Tang, Steve Li, and Richard Liu. Builds tools to automatically red-team and stress-test LLM systems, surfacing failure modes for customers including frontier AI labs.
Non-profit🇺🇸 Cambridge
Institute for Responsible Superintelligence1 person
Develops formal, safe-by-design foundations for superintelligence, including precise safety objectives, mechanisms with analyzable guarantees, and proofs of concept that frontier AI developers can evaluate and adopt.
Non-profit🇩🇪 Berlin
KIRA Center1 person
Independent think tank shaping policy for advanced AI. Its work spans AI safety, European regulation, and Germany's computing capacity; staff have helped write international AI safety reports and the EU general-purpose AI code of practice.
Non-profit🇨🇦 Montreal
LawZero1 person
AI governance and safety organisation co-founded by Yoshua Bengio. Works to develop international frameworks for AI oversight and advocates for responsible AI development policy, with a focus on existential and catastrophic risk.
Company🇺🇸 United States
LeoLabs1 person
Space-safety and orbital-intelligence company that operates radar systems and a data platform for tracking satellites and debris in low Earth orbit.