DeepSeek vs. Guanaco

Overview

DeepSeek currently offers V3 and R1 models, both of which are highly efficient and performant. V3 is comparable to models such as Anthropic's Sonnet 3.5, while R1 is comparable to models such as OpenAI's o1.

DeepSeek is a Chinese startup that began releasing LLMs in 2023 with DeepSeek Coder. In rapid succession, DeepSeek has since released more powerful models, most notably releasing DeepSeek V3 at the end of 2024 and DeepSeek R1 at the beginning of 2025. DeepSeek V3 and R1 set the frontier in terms of efficiency while maintaining high performance. The release of V3 and R1 sent shockwaves through the US technology sector, especially given the low cost with which V3 and R1 were trained (orders of magnitude less than the cost of training equivalent US models.)

Initial release: 2023-11-29

Reference

https://www.deepseek.com/

Further Reading

Overview

Guanaco is an LLM based off the QLoRA 4-bit finetuning method developed by Tim Dettmers et. al. in the UW NLP group. Guanaco achieves 99% ChatGPT performance on the Vicuna benchmark.

Guanaco is an LLM that uses a finetuning method called LoRA that was developed by Tim Dettmers et. al. in the UW NLP group. With QLoRA, it becomes possible to finetune up to a 65B parameter model on a 48GB GPU without loss of performance relative to a 16-bit model. The Guanaco model family outperforms all previously released models on the Vicuna benchmark. However, given the models are based off of the LLaMA model family, commercial use is not permitted.

Initial release: 2023-05-23

Reference

https://github.com/artidoro/qlora

Further Reading

• https://arxiv.org/abs/2305.14314
• https://twitter.com/Tim_Dettmers/status/1661379354507476994