GPTNeo
GPTNeo is a model released by EleutherAI to try and provide an open source model with capabilities similar to OpenAI's GPT-3 model. One of the earliest such models, GPTNeo was trained on The Pile, Eleuther's corpus of web text.
A side-by-side comparison of release history, capabilities, licensing and model sizes.
| Attribute |
 GPTNeo
|
 Koala
|
|---|---|---|
| Developer | EleutherAI, Together | BAIR |
| Current generation | – | – |
| Initial release | 2021-03-21 | 2023-04-03 |
| Instruct tuned | No | Yes |
| Coding | No | No |
| Finetuning | Yes | Yes |
| Open source | Yes | Yes |
| License | Apache 2.0 | Noncommercial |
| Model sizes | 1.3B, 2.7B | 7B, 13B |
| Links |
GPTNeo is a model released by EleutherAI to try and provide an open source model with capabilities similar to OpenAI's GPT-3 model. One of the earliest such models, GPTNeo was trained on The Pile, Eleuther's corpus of web text.
Released alongside Vicuna, Koala is one of many descendants of the Meta LLaMA model trained on dialogue data collected from the web. On the developers' benchmarks, Koala outperforms its sibling Alpaca, though its adoption has been significantly less than that of its other sibling, Vicuna. Due to its use of LLaMA, only research use is permitted.