Renaissance Labs

Atlas / Europe

Community collective · Paris, France

BigScience

A year-long open research workshop coordinated by Hugging Face that produced the 176B-parameter BLOOM multilingual model.

language modelsdatasetsdistributed

Paris, France

Founded
2021
Type
Community collective
Based in
Paris, France
Key papers
4
Featured projects
8

About

BigScience was a volunteer-driven research workshop in which several hundred researchers worked in the open, with Hugging Face coordinating and French government funding supporting it.

The main output was BLOOM, released on 12 July 2022 with 176 billion parameters, 46 natural languages and 13 programming languages. It was trained between March and July 2022 on the Jean Zay supercomputer on the ROOTS corpus.

The code is Apache 2.0 and the weights use the BigScience Responsible AI License. Related releases include xP3 and BLOOMZ.

Mission and ambitions

The workshop aimed to study large language models openly, with participants from many countries sharing data, code and evaluation.

Known for

  1. 01

    BLOOM 176B multilingual model

  2. 02

    ROOTS corpus

  3. 03

    xP3 and BLOOMZ instruction tuning

Milestones

  1. 2021

    T0 prompted training

    Multitask Prompted Training Enables Zero-Shot Task Generalization was published.

  2. 2022

    BLOOM training begins

    Training ran on the Jean Zay supercomputer from March.

  3. 2022

    BLOOM released

    A 176B open model in 46 languages and 13 programming languages.

  4. 2022

    xP3 and BLOOMZ released

    A multilingual instruction dataset and a model tuned on it.

  5. 2023

    ROOTS paper

    The ROOTS corpus paper was presented at NeurIPS 2022 and posted to arXiv in 2023.

Open source

GitHubgithub.com/bigscience-workshop
  • Run LLMs at home, BitTorrent-style; most-starred repo.

  • Toolkit for creating and sharing natural-language prompts.

  • Training code used for BLOOM.

  • Code and data for crosslingual multitask finetuning.

Hugging Facehuggingface.co/bigscience
  • Most downloaded BLOOMZ checkpoint.

  • Small BLOOM base model.

  • P3dataset

    Public Pool of Prompts used to train T0.

  • xP3dataset

    Multilingual prompted dataset for BLOOMZ/mT0.

Research papers

Notable releases

Openness

Open weights, datasets and code, with BLOOM under the Responsible AI License.

People

Listed from the lab's own pages and public profiles.

Contact and community

Contact details and links are on the lab's website (https://bigscience.huggingface.co); code questions go through issues on their GitHub organisation.

Sources

Compiled from the lab's public pages, Hugging Face and GitHub. Last checked 2026-11-17. Something wrong or out of date? Tell us on Discord.

Labs nearby