Natural Language Processing

FastChat is LMSYS's open platform for training, serving and evaluating large language model chatbots. It powers Vicuna and the Chatbot Arena, and exposes an OpenAI-compatible API server for local models.

MLC LLM is a universal deployment engine that compiles and runs large language models natively on almost any hardware — phones, laptops, browsers and servers — using machine-learning compilation built on Apache TVM.

Mistral Small 3 is a 24B open model from Mistral AI built for low latency. Apache-2.0 licensed, it delivers strong reasoning and instruction following at a size that competes with much larger models while staying fast and efficient.

Phi-4 is Microsoft's 14B small language model that delivers reasoning and math performance rivaling far larger models, achieved through heavy use of high-quality synthetic 'textbook' training data. Open weights under MIT.

BERT is Google's landmark bidirectional transformer that reshaped natural language processing. By pretraining on masked-language modelling, it learns deep two-way context and fine-tunes to leading results on understanding tasks.

TensorRT-LLM is NVIDIA's open-source library for optimising large-language-model inference on NVIDIA GPUs. It compiles models into highly tuned engines with quantisation, batching and kernel fusion for maximum throughput and low latency.

R1-1776 is Perplexity AI's open post-trained version of DeepSeek-R1. It preserves the model's strong reasoning while removing built-in censorship, producing unbiased, factual answers on sensitive topics. Released openly under MIT.

Llama 2 is Meta's landmark open large language model, released in 7B, 13B and 70B sizes with chat-tuned variants. Its capable quality and broadly permissive licence made it the foundation of the modern open-LLM ecosystem.

T5 (Text-to-Text Transfer Transformer) is Google's influential encoder-decoder model that frames every NLP task as text-to-text. From translation to summarization to classification, one unified format handles them all.

Yi-34B is a bilingual (English and Chinese) open large language model from 01.AI, strong on reasoning and knowledge for its size, with a 200K-token long-context variant and a permissive licence for commercial use.

Mamba-2 is a selective state-space model (SSM) from CMU and Princeton that matches Transformer quality on language modelling while scaling linearly with sequence length, released openly under Apache 2.0.

DBRX is an open mixture-of-experts large language model from Databricks with 132B total parameters but only 36B active per token, giving strong quality with efficient inference and a 32K context window.

Qwen1.5 72B is a large open language model from Alibaba Cloud. Strongly multilingual with a 32K context, it offers competitive reasoning, coding and chat across many languages, and ships with chat-tuned variants and broad framework support.

OLMo is a truly open language model from the Allen Institute for AI (AI2): open weights, open training code, and the open Dolma dataset. It is built for reproducible, transparent LLM research, with 1B and 7B sizes.