Mistral vs. RedPajama-INCITE

Overview

Developed by some of the researchers behind Llama, the Mistral large language models are the gold standard for accessible and performant open source models.

Mistral AI offers 7B and a mixture-of-experts 8x7B open source models competitive or better than commercial models of similar size. Available under the Apache 2.0 license, the Mistral models are now also available via most cloud vendors. After releasing their open source models, Mistral AI has also begun offering Small, Large, and Embed models via their business API.

Initial release: 2023-09-27

Reference

https://mistral.ai/technology/#models

Further Reading

• https://mistral.ai/news/announcing-mistral-7b/
• https://mistral.ai/news/mixtral-of-experts/

Overview

RedPajama-INCITE is the first family of models trained on the RedPajama base dataset. The goal of the RedPajama-INCITE models is to replicate the LLaMA recipe but make the model fully open source under the Apache license. As of the initial release, the 3B parameter model is best-in-class, with the 7B parameter model in progress. Update as of June 6, 2023: the 7B parameter model was made available, outperforming other models of the same size.

Initial release: 2023-05-05

Reference

https://www.together.xyz/blog/redpajama-models-v1

Further Reading

• https://www.together.xyz/blog/redpajama-7b