Guanaco vs. Kimi

Overview

Guanaco is an LLM based off the QLoRA 4-bit finetuning method developed by Tim Dettmers et. al. in the UW NLP group. Guanaco achieves 99% ChatGPT performance on the Vicuna benchmark.

Guanaco is an LLM that uses a finetuning method called LoRA that was developed by Tim Dettmers et. al. in the UW NLP group. With QLoRA, it becomes possible to finetune up to a 65B parameter model on a 48GB GPU without loss of performance relative to a 16-bit model. The Guanaco model family outperforms all previously released models on the Vicuna benchmark. However, given the models are based off of the LLaMA model family, commercial use is not permitted.

Initial release: 2023-05-23

Reference

https://github.com/artidoro/qlora

Further Reading

• https://arxiv.org/abs/2305.14314
• https://twitter.com/Tim_Dettmers/status/1661379354507476994

Overview

Kimi is Moonshot AI's open-weight family for long-context reasoning, coding, multimodal agents, and research. Kimi K3 is the current generation.

Moonshot AI launched Kimi in 2023 with an early focus on long-context document understanding. Kimi K3, released in July 2026, is a 2.8T-parameter mixture-of-experts model with 104B active parameters. It combines native vision, a one-million-token context window, long-horizon coding, knowledge work, reasoning, and agent orchestration. The full model weights are available under the Kimi K3 License.

Initial release: 2023-10-09

Current generation: Kimi K3

Reference

https://huggingface.co/moonshotai/Kimi-K3

Further Reading

• https://github.com/MoonshotAI/Kimi-K2
• https://github.com/MoonshotAI/Kimi-K3
• https://huggingface.co/moonshotai/Kimi-K3