Get started
LLM Index

MPT vs. RedPajama-INCITE

A side-by-side comparison of release history, capabilities, licensing and model sizes.

Attribute MPT RedPajama-INCITE
Developer MosaicML Together
Current generation – –
Initial release 2023-05-05 2023-05-05
Instruct tuned Yes Yes
Coding No No
Finetuning Yes Yes
Open source Yes Yes
License Apache 2.0 Apache 2.0
Model sizes 7B, 30B 3B, 7B
Links

MPT

MPT-7B and MPT-30B are a set of models that are part of MosaicML's Foundation Series. Trained on 1T tokens, the developers state that MPT-7B matches the performance of LLaMA while also being open source, while MPT-30B outperforms the original GPT-3. In addition to the base model, the developers also offer MPT-Instruct, MPT-Chat, and MPT-7B-StoryWriter-65k+, the last of which is trained on a context length of 65K tokens.

Further reading

RedPajama-INCITE

RedPajama-INCITE is the first family of models trained on the RedPajama base dataset. The goal of the RedPajama-INCITE models is to replicate the LLaMA recipe but make the model fully open source under the Apache license. As of the initial release, the 3B parameter model is best-in-class, with the 7B parameter model in progress. Update as of June 6, 2023: the 7B parameter model was made available, outperforming other models of the same size.

Further reading

Compare other models

Other MPT comparisons

Other RedPajama-INCITE comparisons