The Cerebras-GPT family of models was developed by the AI accelerator company Cerebras following Chinchilla scaling laws as a demonstration of its Wafter-Scale Cluster technology.
Initial release: 2023-03-28
MPT-7B and MPT-30B are a set of models that are part of MosaicML's Foundation Series. Trained on 1T tokens, the developers state that MPT-7B matches the performance of LLaMA while also being open source, while MPT-30B outperforms the original GPT-3. In addition to the base model, the developers also offer MPT-Instruct, MPT-Chat, and MPT-7B-StoryWriter-65k+, the last of which is trained on a context length of 65K tokens.
Initial release: 2023-05-05
|Products & Features|
|License||Apache 2.0||Apache 2.0|
|Model Sizes||1.3B, 2.7B, 6.7B, 13B||7B, 30B|