MLC LLM

MLC LLM is a machine learning compiler and high-performance deployment engine for large language models. The mission of this project is to enable everyone to develop, optimize, and deploy AI models natively on everyone’s platforms.

Because the models run locally, it only works for the devices with sufficient VRAM depending on the models being used.

MLC LLM allows any language model to be deployed natively on a diverse set of hardware backends and native applications. It allows you to run open-language models downloaded from the internet. Each model can be subject to its respective licenses.

MLC LLM can also be used in a web browser and brings language model inference directly onto web browsers with hardware acceleration. Everything runs inside the browser with no server support and is accelerated with WebGPU. Chat in your browser at https://chat.webllm.ai/