Scientific AI

MLC-LLM is a universal and open-source framework for deploying large language models across various edge devices, enabling effective and rapid inference.

TensorRT-LLM is an open-source library by NVIDIA that delivers highly optimized inference for large language models.