Guanaco vs. Qwen

Overview

Guanaco is an LLM based off the QLoRA 4-bit finetuning method developed by Tim Dettmers et. al. in the UW NLP group. Guanaco achieves 99% ChatGPT performance on the Vicuna benchmark.

Guanaco is an LLM that uses a finetuning method called LoRA that was developed by Tim Dettmers et. al. in the UW NLP group. With QLoRA, it becomes possible to finetune up to a 65B parameter model on a 48GB GPU without loss of performance relative to a 16-bit model. The Guanaco model family outperforms all previously released models on the Vicuna benchmark. However, given the models are based off of the LLaMA model family, commercial use is not permitted.

Initial release: 2023-05-23

Reference

https://github.com/artidoro/qlora

Further Reading

• https://arxiv.org/abs/2305.14314
• https://twitter.com/Tim_Dettmers/status/1661379354507476994

Overview

Qwen is Alibaba Cloud's family of large language models, including chat models, code models, and multimodal variants. Qwen 3 and QwQ are the latest releases, with strong reasoning capabilities.

Qwen (formerly Tongyi Qianwen) is a comprehensive family of large language models developed by Alibaba Cloud. The series includes base models, chat models, code-specific variants (CodeQwen), and multimodal models (Qwen-VL). Qwen models excel at multilingual tasks and have become particularly popular in Asia. The latest releases include Qwen 3 and QwQ (focused on reasoning), competing effectively with Western counterparts while offering strong Chinese language support.

Initial release: 2023-09-13

Reference

https://qwenlm.github.io/

Further Reading

• https://arxiv.org/abs/2309.16609
• https://huggingface.co/Qwen