Cerebras-GPT vs. Phi

Overview

The Cerebras-GPT family of models was developed by the AI accelerator company Cerebras following Chinchilla scaling laws as a demonstration of its Wafter-Scale Cluster technology.

Initial release: 2023-03-28

Reference

https://huggingface.co/cerebras

Further Reading

Overview

Phi is a series of compact language models developed by Microsoft using high-quality synthetic data and curated web content. Phi-3 and Phi-3.5 (2024) significantly raised the bar for small models, with the 3.8B Phi-3 Mini matching much larger open-source models on key benchmarks.

Phi-1 and Phi-2 are 1.3B and 2.7B parameter language models, respectively, developed by Microsoft to demonstrate the ability of smaller language models trained on high-quality data. In April 2024, Phi-3 was released across Mini (3.8B), Small (7B), and Medium (14B) sizes. Phi-3 Mini achieves performance competitive with Llama 3 8B and Mistral 7B at a fraction of the compute, making it ideal for edge and constrained-resource deployments. Phi-3.5 Mini and MoE followed in August 2024 with further improvements.

Initial release: 2023-06-20

Reference

https://azure.microsoft.com/en-us/products/phi

Further Reading

• https://huggingface.co/microsoft/phi-2
• https://arxiv.org/abs/2306.11644
• https://huggingface.co/microsoft/Phi-3-mini-4k-instruct
• https://huggingface.co/microsoft/Phi-3.5-mini-instruct