Gemma 2 vs. Grok

Overview

Gemma 2 succeeds the Gemma family of lightweight open models from Google built using the same processes used for the the larger Gemini models.

Gemma 2 is the successor to the Gemma family of open models, including larer models (9B and 27B parameters) with outsized performance across benchmarks. Using a combination of techniques such as training on twice as much data, knowledge distillation, and architectural improvements such as sliding window attention, logit soft-capping, and model merging, Gemma 2 outperforms models of similar size (such as Llama 3), with the 27B parameter model being competitive with models more than twice its size (such as Llama 70B).

Initial release: 2024-06-27

Reference

https://blog.google/technology/developers/google-gemma-2/

Further Reading

• https://ai.google.dev/gemma/docs
• https://huggingface.co/blog/gemma2

Overview

Grok is a mixture-of-experts model offered by xAI.

Grok is the flagship language model of xAI, the AI company founded by Elon Musk with close ties to X.com (FKA Twitter). Grok-1 was initialized released late in 2023 and later open-sourced in March of 2024. On March 28, 2024, Grok-1.5 was released with significantly better performance and a context length of 128,000 tokens.

Initial release: 2023-11-05

Reference

https://huggingface.co/xai-org/grok-1

Further Reading

• https://github.com/xai-org/grok-1/tree/main