Orca vs. Phi

Overview

Orca is a descendant of LLaMA developed by Microsoft with finetuning on explanation traces obtained from GPT-4.

Orca-13B is a LLM developed by Microsoft. It is based on LLaMA with finetuning on complex explanation traces obtained from GPT-4. By using rich signals, Orca surpasses the performance of models such as Vicuna-13B on complex tasks. However, given its model backbone and the data used for its finetuning, Orca is under noncommercial use.

Initial release: 2023-06-05

Reference

https://aka.ms/orca-lm

Further Reading

• https://arxiv.org/pdf/2306.02707.pdf

Overview

Phi is a series of compact language models developed by Microsoft using high-quality synthetic data and curated web content. Phi-3 and Phi-3.5 (2024) significantly raised the bar for small models, with the 3.8B Phi-3 Mini matching much larger open-source models on key benchmarks.

Phi-1 and Phi-2 are 1.3B and 2.7B parameter language models, respectively, developed by Microsoft to demonstrate the ability of smaller language models trained on high-quality data. In April 2024, Phi-3 was released across Mini (3.8B), Small (7B), and Medium (14B) sizes. Phi-3 Mini achieves performance competitive with Llama 3 8B and Mistral 7B at a fraction of the compute, making it ideal for edge and constrained-resource deployments. Phi-3.5 Mini and MoE followed in August 2024 with further improvements.

Initial release: 2023-06-20

Reference

https://azure.microsoft.com/en-us/products/phi

Further Reading

• https://huggingface.co/microsoft/phi-2
• https://arxiv.org/abs/2306.11644
• https://huggingface.co/microsoft/Phi-3-mini-4k-instruct
• https://huggingface.co/microsoft/Phi-3.5-mini-instruct