Renaissance Labs

Atlas / North America

Nonprofit · San Francisco, United States

Center for AI Safety

A San Francisco nonprofit reducing societal-scale AI risk through research, field-building and open benchmarks such as HarmBench and WMDP.

safetyevaluationdatasets

San Francisco, United States

Founded
2022
Type
Nonprofit
Based in
San Francisco, United States
Key papers
4
Featured projects
8

About

The Center for AI Safety was started in 2022 and is led by Dan Hendrycks as executive and research director, with Oliver Zhang as managing director. Its stated purpose is to reduce societal-scale risks from artificial intelligence, and it organizes its staff into research, cloud and DevOps, projects and operations teams.

CAIS has operated a compute cluster since March 2023 that gives university safety researchers access to large models, and it has run machine learning safety workshops, a Trojan detection competition and a philosophy fellowship. In May 2023 it published the Statement on AI Risk, signed by more than 600 researchers and public figures.

Its research output includes widely used open evaluations. HarmBench standardizes tests of automated red teaming and refusal, WMDP measures hazardous knowledge and evaluates unlearning methods, and Humanity's Last Exam collects expert-written questions to measure frontier capability.

Mission and ambitions

CAIS works to reduce societal-scale risks from AI through technical research, field-building and advocacy.

Known for

  1. 01

    Statement on AI Risk

  2. 02

    HarmBench red-teaming benchmark

  3. 03

    WMDP and Humanity's Last Exam benchmarks

  4. 04

    Compute cluster for safety researchers

Milestones

  1. 2022

    Center founded

    Established as a nonprofit focused on societal-scale AI risk.

  2. 2023

    Compute cluster

    Began providing GPU access to university safety researchers in March 2023.

  3. 2023

    Statement on AI Risk

    Signed by more than 600 AI experts and public figures.

  4. 2024

    HarmBench

    Open framework for evaluating automated red teaming and robust refusal.

  5. 2024

    WMDP benchmark

    Open benchmark for hazardous knowledge and unlearning, with dozens of co-authors across institutions.

  6. 2025

    Humanity's Last Exam

    Multimodal expert-written benchmark built with Scale AI and over a thousand contributors.

Open source

GitHubgithub.com/centerforaisafety
Hugging Facehuggingface.co/cais

Research papers

Notable releases

Openness

Open benchmarks and code.

People

  • Dan HendrycksExecutive and research director

    Creator of MMLU and the GELU activation.

  • Oliver ZhangManaging director

    Listed on the CAIS leadership page.

Listed from the lab's own pages and public profiles.

Contact and community

Contact details and links are on the lab's website (https://safe.ai); code questions go through issues on their GitHub organisation.

Sources

Compiled from the lab's public pages, Hugging Face and GitHub. Last checked 2026-11-17. Something wrong or out of date? Tell us on Discord.

Labs nearby