Phi vs. Vicuna

Overview

Phi is a series of compact language models developed by Microsoft using high-quality synthetic data and curated web content. Phi-3 and Phi-3.5 (2024) significantly raised the bar for small models, with the 3.8B Phi-3 Mini matching much larger open-source models on key benchmarks.

Phi-1 and Phi-2 are 1.3B and 2.7B parameter language models, respectively, developed by Microsoft to demonstrate the ability of smaller language models trained on high-quality data. In April 2024, Phi-3 was released across Mini (3.8B), Small (7B), and Medium (14B) sizes. Phi-3 Mini achieves performance competitive with Llama 3 8B and Mistral 7B at a fraction of the compute, making it ideal for edge and constrained-resource deployments. Phi-3.5 Mini and MoE followed in August 2024 with further improvements.

Initial release: 2023-06-20

Reference

https://azure.microsoft.com/en-us/products/phi

Further Reading

• https://huggingface.co/microsoft/phi-2
• https://arxiv.org/abs/2306.11644
• https://huggingface.co/microsoft/Phi-3-mini-4k-instruct
• https://huggingface.co/microsoft/Phi-3.5-mini-instruct

Overview

Released alongside Koala, Vicuna is one of many descendants of the Meta LLaMA model trained on dialogue data collected from the ShareGPT website. According to the authors, Vicuna achieves more than 90% of ChatGPT's quality in user preference tests, while vastly outperforming Alpaca. As of May 2023, Vicuna seems to be the heir apparent of the instruct-finetuned LLaMA model family, though it is also restricted from commercial use.

Initial release: 2023-03-30

Reference

https://vicuna.lmsys.org/

Further Reading

• https://lmsys.org/blog/2023-03-30-vicuna/