Get started
LLM Index

Orca vs. Pythia

A side-by-side comparison of release history, capabilities, licensing and model sizes.

Attribute Orca Pythia
Developer Microsoft EleutherAI, Together
Current generation – –
Initial release 2023-06-05 2023-02-13
Instruct tuned Yes No
Coding No No
Finetuning Yes Yes
Open source Yes Yes
License Noncommercial Apache 2.0
Model sizes 13B 1B, 1.4B, 2.8B, 6.9B, 12B
Links

Orca

Orca is a descendant of LLaMA developed by Microsoft with finetuning on explanation traces obtained from GPT-4.

Orca-13B is a LLM developed by Microsoft. It is based on LLaMA with finetuning on complex explanation traces obtained from GPT-4. By using rich signals, Orca surpasses the performance of models such as Vicuna-13B on complex tasks. However, given its model backbone and the data used for its finetuning, Orca is under noncommercial use.

Further reading

Pythia

The most recent (as of May 2023) effort from EleutherAI, Pythia is a set of LLMs trained on The Pile. While it appears to outperform OPT and GPTNeo, its performance against GPT-J is unclear. Versions of Pythia have also been instruct-tuned by the team at Together.

Compare other models

Other Orca comparisons

Other Pythia comparisons