Mistral vs. Orca

Overview

Developed by some of the researchers behind Llama, the Mistral large language models are the gold standard for accessible and performant open source models.

Mistral AI offers 7B and a mixture-of-experts 8x7B open source models competitive or better than commercial models of similar size. Available under the Apache 2.0 license, the Mistral models are now also available via most cloud vendors. After releasing their open source models, Mistral AI has also begun offering Small, Large, and Embed models via their business API.

Initial release: 2023-09-27

Reference

https://mistral.ai/technology/#models

Further Reading

• https://mistral.ai/news/announcing-mistral-7b/
• https://mistral.ai/news/mixtral-of-experts/

Overview

Orca is a descendant of LLaMA developed by Microsoft with finetuning on explanation traces obtained from GPT-4.

Orca-13B is a LLM developed by Microsoft. It is based on LLaMA with finetuning on complex explanation traces obtained from GPT-4. By using rich signals, Orca surpasses the performance of models such as Vicuna-13B on complex tasks. However, given its model backbone and the data used for its finetuning, Orca is under noncommercial use.

Initial release: 2023-06-05

Reference

https://aka.ms/orca-lm

Further Reading

• https://arxiv.org/pdf/2306.02707.pdf