Project Maya Puts a 748B MoE on Home GPUs by Moving Experts Between GPU, RAM and SSD

What changed

• v1.1.47 (November 1): a second version of the DeepSeek Maya-XS quant. The notes call it "about 12% closer to the original model (KLD 0.251 against 0.284)" and "about 20% faster at writing, at almost the same size (117 GB)". KLD measures how far the quantized model's next-token probabilities drift from the original's; lower is better. Unsloth's DeepSeek quants now share Maya's Q6_K n-gram tables on disk, so the Q8 download drops "from 508 to 467 GB".
• v1.1.48 (November 1): when experts are offloaded across several GPUs, "prompts are read in 4K-token pieces by default", which "leaves room for about 3.5% more experts on the GPUs". The notes report that the engine "reads an 8K prompt about 24-30% faster on 3x V100". A fix to the DeepSeek thinking budget means "the next turn no longer reads the whole thinking again (59 s at 32K tokens before)".
• v1.1.49 (November 2): fixes a start-up check that could stop with "changed after the pack was built". Running setup once now "turns that old copy into a link to the shared one, giving back about 161 GB." It also reserves memory for the first request on 8 GB cards.

How the three tiers work

The README explains it this way: the engine "keeps the experts your conversation uses most on the GPU, the next ones in RAM", with "the rest on your NVMe SSD, and moves them as you chat." For DeepSeek it lists Maya-XS at 277 GB and Maya-S at 340 GB, and notes that "DeepSeek's sizes include its Engram tables". Maya-XS fits "the whole model in a 128 GB PC's memory (the Engram tables stay on the SSD)". The engine "pins most of the free RAM (all but about 6 GB) for its expert tier".

The next two calculations are ours, not the project's, and we have not run the engine. The 277 GB figure is consistent with about 117 GB of model files plus about 160 GB of n-gram tables, which matches the 161 GB that v1.1.49 frees by removing a duplicate copy. A single 24 GB card can hold at most about a fifth of those 117 GB (24 divided by 117 is about 0.21), and that is before the KV cache and activations take their share. Most experts therefore live in RAM, and speed depends on how often the experts a token needs are already on the GPU.

What remains uncertain. The KLD, speed and accuracy figures come from the maintainers. The README says the decode speeds in its hardware table are reported by users. For most of these claims the release notes name neither the hardware nor the method. The AMD, Windows and Pascal paths are marked experimental. Four releases in a day is fast, and setups can break between versions: v1.1.48 had to fix --engine strata, which v1.1.47 had pointed at the test engine.

Analysis: Spreading experts across tiers turns "the model doesn't fit" into a caching problem, and caches behave well only when the workload is predictable. Here is a hypothetical example. Suppose most tokens in the Belgavi's Book Store help desk go through the experts that handle returns, refunds and order status. A cache sized for those experts runs fast on a normal day. A burst of unusual CASE tickets, such as a batch of damaged-shipment claims in another language, could make the engine keep swapping experts in and out of GPU memory. Before adopting an engine like this, Tanishka could replay a normal day of help-desk prompts, then a shuffled set of rare topics, and compare the decode speed of the two runs. If the second run is much slower, size the hardware for the worst hour, not the average day.

Sources and evidence

• Project Maya README (mw00/project-maya, GitHub)
• Project Maya v1.1.47 release notes (GitHub, November 1)
• Project Maya v1.1.48 release notes (GitHub, November 1)
• Project Maya v1.1.49 release notes (GitHub, November 2)