LLM Libraries
A rapidly growing and evolving ecosystem of libraries and SDKs for working with large language models (LLMs) has emerged. This page includes some popular libraries for training, evaluating, finetuning, optimizing, connecting, serving, and developing interfaces for LLMs.
| Library | Tags | Description |
|---|---|---|
| Alpaca Farm | AlpacaFarm is a framework for research and development of systems that learn from human feedback (such as instruction-following LLMs). It contains code for simulating preference feedback, automated evaluation, and reinforcement learning algorithms such as PPO. | |
| Flax | Flax is a neural network library for JAX(a library that provides composable differentiation and vectorization operations for the Python ecosystem) that is designed for flexibility. | |
| GGML | GGML is a tensor framework for enabling large models to be run on commodity software. It provides integer quantization of models, support for different platforms and intrinsics, and web support. Popular libraries based on GGML include llama.cpp and whisper.cpp. | |
| Hugging Face | Hugging Face is the Github for machine learning. Hugging Face provides the popular Transformers library for working with the Transformer model, as well as as its hub for machine learning models and datasets. | |
| Lamini | Lamini is an LLM platform that allows developers to build and run their own custom, private LLMs. Capabilities include fine-tuning, RLHF, and optimizations so the self-hosted LLM runs efficiently. | |
| LangChain | LangChain is a framework for developing applications that use language models. LangChain provides modules for models, prompts, indices, chains, and agents, with the ability to "chain" these modules together. | |
| LlamaIndex | LlamaIndex (fka GPT Index) is a framewor for augmenting LLMs with private/custom data. It offers data connectors, indices/graphs that are LLM-compatible, and a query interface over the data. | |
| LMFlow is an extensible toolbox for finetuning large language models, including LLMs. It supports common backbones such as LLaMA and GPT-2. | ||
| MLC LLM | MLC LLM allows language models to be optimized and deployed natively on a broad set of hardware and native applications. Supported platforms include iOS, Android, Apple Silicon, AMD, Intel, NVIDIA, and WebGPU. |