DeepSeek vs. Guanaco

Overview

DeepSeek develops open-weight mixture-of-experts models for reasoning, coding, agents, and efficient long-context inference. DeepSeek V4 is the current generation.

DeepSeek began releasing language models in 2023 with DeepSeek Coder, then gained broad adoption with DeepSeek V3 and the R1 reasoning model in 2024 and 2025. DeepSeek V4, released in April 2026, includes a 284B-parameter Flash model with 13B active parameters and a 1.6T-parameter Pro model with 49B active parameters. Both support one-million-token context windows and are released with open weights under the MIT license.

Initial release: 2023-11-29

Current generation: DeepSeek V4

Reference

https://www.deepseek.com/

Further Reading

• https://api-docs.deepseek.com/news/news260424/
• https://huggingface.co/collections/deepseek-ai/deepseek-v4

Overview

Guanaco is an LLM based off the QLoRA 4-bit finetuning method developed by Tim Dettmers et. al. in the UW NLP group. Guanaco achieves 99% ChatGPT performance on the Vicuna benchmark.

Guanaco is an LLM that uses a finetuning method called LoRA that was developed by Tim Dettmers et. al. in the UW NLP group. With QLoRA, it becomes possible to finetune up to a 65B parameter model on a 48GB GPU without loss of performance relative to a 16-bit model. The Guanaco model family outperforms all previously released models on the Vicuna benchmark. However, given the models are based off of the LLaMA model family, commercial use is not permitted.

Initial release: 2023-05-23

Reference

https://github.com/artidoro/qlora

Further Reading

• https://arxiv.org/abs/2305.14314
• https://twitter.com/Tim_Dettmers/status/1661379354507476994