Get started
LLM Index

Falcon vs. Pythia

A side-by-side comparison of release history, capabilities, licensing and model sizes.

Attribute Falcon Pythia
Developer TII EleutherAI, Together
Current generation – –
Initial release 2023-05-23 2023-02-13
Instruct tuned Yes No
Coding No No
Finetuning Yes Yes
Open source Yes Yes
License Apache 2.0 Apache 2.0
Model sizes 7B, 40B 1B, 1.4B, 2.8B, 6.9B, 12B
Links

Falcon

Falcon LLM is the flagship LLM of the Technology Innovation Institute in Abu Dhabi.

Falcon LLM is a powerful LLM developed by the Technology Innovation Institute (https://www.tii.ae). Unlike other popular LLMs, Falcon was not built off of LLaMA, but instead using a custom data pipeline and distributed training system. The dataset is the RefinedWeb dataset (available on Hugging Face), and the initial models are available in 7B and 40B forms. Instruct and chat-finetuned models are available as well. Perhaps most excitingly, Falcon outperforms LLaMA, the previous de facto pretrained LLM.

Further reading

Pythia

The most recent (as of May 2023) effort from EleutherAI, Pythia is a set of LLMs trained on The Pile. While it appears to outperform OPT and GPTNeo, its performance against GPT-J is unclear. Versions of Pythia have also been instruct-tuned by the team at Together.

Compare other models

Other Falcon comparisons

Other Pythia comparisons