Renaissance Labs

Atlas / North America

Nonprofit · Alexandria, United States

EleutherAI

EleutherAI is a non-profit research institute that grew from a Discord server into a leading source of open language models, datasets and evaluation tools.

language modelsevaluationinterpretabilitydatasetsdistributed

Alexandria, United States

Founded
2020
Type
Nonprofit
Based in
Alexandria, United States
Key papers
4
Featured projects
8

About

EleutherAI began in July 2020 as a Discord server under the working name LibreAI. Connor Leahy, Leo Gao and Sid Black are named as founders, and the group was renamed EleutherAI later that month. In early 2023 it was incorporated as a non-profit research institute.

It first used compute from Google's TPU Research Cloud and later from CoreWeave. The group released The Pile in December 2020, followed by the GPT-Neo, GPT-J and GPT-NeoX-20B models, and the Pythia suite for studying training dynamics.

Today it reports about fifteen full- and part-time staff along with regular volunteers. Its lm-evaluation-harness, the most starred repository in its GitHub organization, is used widely to benchmark language models.

Mission and ambitions

To advance interpretability and alignment research on open foundation models, keep such models open to study beyond a few companies, and educate the public about their capabilities and risks.

Known for

  1. 01

    The Pile dataset

  2. 02

    GPT-Neo, GPT-J and GPT-NeoX-20B

  3. 03

    lm-evaluation-harness and Pythia

Milestones

  1. 2020

    The Pile released

    An 800GB dataset of diverse text for language modeling, authored by Gao, Biderman, Black and others.

  2. 2021

    GPT-Neo

    The first GPT-Neo models of 125M, 1.3B and 2.7B parameters were published in March 2021.

  3. 2021

    GPT-J-6B

    A six-billion-parameter open model was released in June 2021.

  4. 2022

    GPT-NeoX-20B

    A 20-billion-parameter model trained with CoreWeave compute was released in February 2022.

  5. 2023

    Pythia suite and incorporation

    Pythia provided checkpoints for analyzing models across training and scaling, and the group incorporated as a non-profit institute.

  6. 2025

    Common Pile v0.1

    A training dataset built to exclude copyrighted material was released in June 2025.

Open source

GitHubgithub.com/EleutherAI
  • Most-starred repo; the standard few-shot LLM evaluation framework.

  • Model-parallel GPT-2/GPT-3-style training library from the early GPT-Neo era.

  • Large-scale autoregressive training library behind GPT-NeoX-20B and Pythia.

  • Hub for the Pythia suite and learning-dynamics research.

Hugging Facehuggingface.co/EleutherAI

Research papers

Notable releases

Openness

Open weights, datasets and code.

People

Listed from the lab's own pages and public profiles.

Contact and community

Contact details and links are on the lab's website (https://www.eleuther.ai); code questions go through issues on their GitHub organisation.

Public Discord linked from the lab's own website for community discussion and announcements.

Sources

Compiled from the lab's public pages, Hugging Face and GitHub. Last checked 2026-11-17. Something wrong or out of date? Tell us on Discord.

Labs nearby