
Academic lab · Tokyo, Japan
Swallow LLM
Swallow is a Japanese LLM project at the Institute of Science Tokyo that continually pretrains open models to give them strong Japanese ability.
Tokyo, Japan
- Founded
- 2023
- Type
- Academic lab
- Based in
- Tokyo, Japan
- Key papers
- 3
- Featured projects
- 8
About
Swallow is led by researchers in the Okazaki and Yokota laboratories at the Institute of Science Tokyo, together with the National Institute of Advanced Industrial Science and Technology (AIST). Naoaki Okazaki leads the project and the pre-training corpus team, Rio Yokota leads LLM training, and Sakae Mizuki leads instruction tuning and evaluation.
The project adapts existing open models such as Llama, Mistral, Gemma, Qwen and gpt-oss to Japanese through continual pretraining and instruction tuning. Its model list runs from the first Swallow models to Llama 3.3 Swallow 70B, Qwen3 Swallow and GPT-OSS Swallow.
Weights are published on Hugging Face under the tokyotech-llm organization, and the project documents its corpus and evaluation work for the Japanese community.
Mission and ambitionsSwallow aims to build and openly release Japanese large language models and the corpora and evaluation methods needed to improve them.
Known for
- 01
Japanese continual-pretrained Llama models
- 02
Swallow corpus and evaluation work
- 03
Llama 3.3 Swallow 70B
Milestones
- 2023
First Swallow models
Llama-based Swallow models extend Japanese ability of open LLMs.
- 2024
Llama 3 Swallow
Models built on Llama 3 are released.
- 2024
Llama 3.1 Swallow
The line is updated to Llama 3.1.
- 2025
Llama 3.3 Swallow 70B
A 70B model is added to the family.
Open source
Evaluation framework for post-trained models.
Evaluation scripts.
Corpus code.
Deduplication tool.
Japanese Llama 3.
Japanese Llama 3.1.
- swallow-code-v2dataset
Code dataset.
- swallow-math-v2dataset
Math dataset.
Research papers
Notable releases
Openness
Weights on Hugging Face, code on GitHub.
People
- Naoaki OkazakiProfessor; project leader
- Rio YokotaProfessor; LLM training team leader
- Sakae MizukiResearcher; instruction tuning and evaluation leader
- Hiroya TakamuraAIST team leader; project manager
Listed from the lab's own pages and public profiles.
Contact and community
Contact via the project website.
Sources
Compiled from the lab's public pages, Hugging Face and GitHub. Last checked 2026-11-17. Something wrong or out of date? Tell us on Discord.



