Before you load a model
What's about to happen
Clicking "Load model" downloads a real set of model weights (a few hundred MB, depending on which you pick) from a CDN and runs them on your GPU using
WebGPU compute shaders, entirely inside this browser tab. Nothing is sent to any server -- there's no backend behind this page at all. The download is cached by your browser, so a second visit loads instantly.
Starting…
Chat
What you're seeing
This page uses MLC's WebLLM, which compiles small open-weight models to run as WebGPU compute shaders -- the same kind of GPU-parallel matrix multiplication a server-side inference engine does, except the "server" is your graphics card and the request never leaves your machine.
- Model size vs. quality: these are small models (well under 1B–3B parameters) chosen so the download is reasonable. They're noticeably less capable than a large hosted model -- expect shorter, sometimes less accurate answers. That trade-off is the whole point of the demo: this is what "on-device" actually costs today.
- First load is slow, second load is instant: the browser caches the downloaded weights, so reloading this page (or coming back later) skips the download and only re-initializes the GPU pipeline.
- Why it's gated behind a button: a few hundred MB is a real cost on a metered connection, so nothing downloads until you explicitly ask for it -- unlike every other lab on this site, which is instant because it's just JavaScript.
- If nothing loads: WebGPU is still rolling out. It needs a modern Chrome or Edge (or Firefox Nightly) on desktop with a real GPU -- it will not work in most mobile browsers, older browsers, or a headless/automated environment.