Gemma 2 vs. Llama 3

Overview

Gemma 2 succeeds the Gemma family of lightweight open models from Google built using the same processes used for the the larger Gemini models.

Gemma 2 is the successor to the Gemma family of open models, including larer models (9B and 27B parameters) with outsized performance across benchmarks. Using a combination of techniques such as training on twice as much data, knowledge distillation, and architectural improvements such as sliding window attention, logit soft-capping, and model merging, Gemma 2 outperforms models of similar size (such as Llama 3), with the 27B parameter model being competitive with models more than twice its size (such as Llama 70B).

Initial release: 2024-06-27

Reference

https://blog.google/technology/developers/google-gemma-2/

Further Reading

• https://ai.google.dev/gemma/docs
• https://huggingface.co/blog/gemma2

Overview

Llama 3 is Meta AI's open source LLM available for both research and commercial use cases (assuming you have less than 700 million monthly active users).

The successor to Llama 2, Llama 3 demonstrates state-of-the-art performance on benchmarks and is, according to Meta, the "best open source models of their class, period". Improvements to the pretraining -- 7X more data than Llama 2 --- and post-training -- careful curation of instruction-tuning data -- processes result in improved alignment and output quality

Initial release: 2024-04-18

Reference

https://llama.meta.com/

Further Reading

• https://ai.meta.com/blog/meta-llama-3/