Dolly vs. Gemma 2

Overview

Dolly is an LLM trained using the Databricks machine learning platform. Originally released without instruct-finetuning, Dolly v2 included tuning on the Stanford Alpaca dataset.

Initial release: 2023-03-24

Reference

https://www.databricks.com/blog/2023/04/12/dolly-first-open-commercially-viable-instruction-tuned-llm

Further Reading

Overview

Gemma 2 succeeds the Gemma family of lightweight open models from Google built using the same processes used for the the larger Gemini models.

Gemma 2 is the successor to the Gemma family of open models, including larer models (9B and 27B parameters) with outsized performance across benchmarks. Using a combination of techniques such as training on twice as much data, knowledge distillation, and architectural improvements such as sliding window attention, logit soft-capping, and model merging, Gemma 2 outperforms models of similar size (such as Llama 3), with the 27B parameter model being competitive with models more than twice its size (such as Llama 70B).

Initial release: 2024-06-27

Reference

https://blog.google/technology/developers/google-gemma-2/

Further Reading

• https://ai.google.dev/gemma/docs
• https://huggingface.co/blog/gemma2