Before you load a model
Chat
What you're seeing
This page uses MLC's WebLLM for local WebGPU inference. The application does not call a hosted chat endpoint. Third-party scripts still execute on this page, so local inference alone is not a guarantee of confidentiality.
- Model size vs. quality: the menu filters for small model families, but download size, memory demand and answer quality vary by model and quantization. Do not treat a successful load as evidence that its answers are correct.
- Cache is an optimization, not a guarantee: weights may be reused, but browser storage can be evicted and GPU initialization still takes time. Unload releases the engine without deleting downloaded cache entries.
- Why weights are gated behind a button: model downloads can be expensive on a metered connection. The runtime library and ordinary page assets can load earlier; model weights are requested only after Load model.
- If nothing loads: this tool checks for a WebGPU adapter, but a successful adapter check does not guarantee sufficient memory, shader support or model compatibility. Browser policy, driver support and available resources matter more than assuming support from the device name alone.
Run a real local model
This tool loads a quantized language model and generates text through WebGPU. It does not replay stored answers or call a hosted chat API. A supported adapter, sufficient memory, compatible shaders and reachable model assets are all required. The initial adapter check is therefore only a prerequisite, not proof that a model will load. The model list comes from the pinned WebLLM runtime and is filtered for small model families; different quantization variants still have different memory and download costs.
Use the model lifecycle controls
Local model chooses a catalog entry before loading. Load model begins the weight download and GPU initialization; the progress text reports library progress, not a guaranteed completion time. After loading, enter a message and press Send, or Enter without Shift. Shift plus Enter inserts a newline. Stop generation requests interruption and waits for streaming to settle before other model operations are enabled. A nonempty partial reply remains part of the conversation. An empty or failed reply does not append an unmatched user turn to the next request.
Reset deliberately
Clear conversation clears the displayed messages and resets model context without unloading the weights. Unload releases the active engine and disables chat; it does not erase downloaded browser cache entries. Loading, generation, reset and unload are serialized so that one operation cannot dispose another operation's engine. Inputs are limited to two thousand characters, generation to two hundred fifty-six tokens, and a conversation to eight completed exchanges. These application bounds are not a tokenizer or a guarantee that every conversation fits the selected model context. If a context or memory error occurs, clear the conversation or unload and retry with a smaller request.
Try and evaluate safely
Ask the model to rewrite a short fictional sentence, then send a follow-up referring to it. Clear conversation and ask the same follow-up: the earlier exchange should no longer be in the request. Start another response and press Stop generation; inspect the partial text before unloading. Compare a fresh load with a cached load, but do not assume the latter is instant because GPU initialization and cache eviction still matter. The runtime, weights, analytics and advertising use external resources, and third-party scripts execute on this page. Do not enter secrets or personal information. Local inference is not an audited confidentiality guarantee; a fluent response is not evidence of factual correctness.