Janas: A Dependency-Free C Engine Runs an 80B Mixture-of-Experts Model on a 32 GB Laptop by Streaming Experts From the SSD

Janas, by Maurizio Cammalleri (prabanta-dev on GitHub), was posted to Hacker News on October 12, 2026 (UTC) as "Run LLMs larger than RAM by streaming experts from SSD". It drew 220 points and 94 comments. The README's headline claim: "An 80-billion parameter model runs, and answers, on a laptop with 32 GB of RAM."

How it works

Janas-LLM is written in C with no dependencies beyond the C library. It targets mixture-of-experts models, where each token only activates a few experts. The weights stay on the NVMe drive. For each token the engine reads the experts it needs from disk with direct I/O, and keeps frequently reused experts in an in-memory cache sized to the free RAM when the model opens. A --cache option sets a hard memory limit. The project also ships a chat client, an OpenAI-compatible server, an MCP server and a benchmark tool. It is licensed under GPL v3 or later, and models come in its own .jns format on Hugging Face.

The numbers, and their limits

All figures come from one laptop: an Intel Core Ultra 9 185H with integrated Arc graphics, 32 GB of DDR5 and an NVMe drive reading about 5.7 GB/s. On Qwen3-Next-80B-A3B, a 48.4 GB file at Q4_K_M, the README reports 62 tokens/s for prompt reading against 16 for llama.cpp's Vulkan backend, and 24 tokens/s for generation against 2.8 for llama.cpp on the CPU alone (2.0 with the GPU). On models that fit in memory the reading gap is much smaller: Qwen3-30B-A3B reads at 135 against 122. The author is open about the limits. Janas read prompts more slowly than llama.cpp on a 3-bit Qwen3-30B quant, though it wrote three times as fast. The Qwen3-Next figures are single runs. The two tools measure slightly differently. And on other hardware, "it is untested: it may be slower, it may refuse the model, it may misbehave." The README is plain about status too: "This is an experiment, not a product."

What the thread argued about

The Hacker News discussion was split. One commenter wrote that "Every few weeks, someone stumbles on what llama.cpp has had for years again, and reimplements it." Others said the hard part is deciding which experts to keep in memory, calling the caching algorithm "absolutely vital, non-obvious, model-dependent". Another pointed out that even fast PCIe 5.0 SSDs are far slower than GPU memory, which caps how far the approach can go.

What remains uncertain. No independent benchmarks yet, one test machine, and a format conversion step that ties users to the project's own model files.

Analysis: Whatever happens to this project, the idea is sound. Mixture-of-experts models touch a small, skewed subset of their weights per token, so the right question for local inference is how fast your SSD and cache are, not how much RAM you have. For teams running private models on developer laptops or edge boxes, this widens the range of usable models. Benchmark expert caching on your own prompts before relying on it, because the cache hit rate, and so the speed, depends on the workload.