# llama.cpp reaches 1.0 with expert caching for MoE models larger than GPU memory

> llama.cpp and ggml have reached 1.0, and the llama.cpp release adds a cache for frequently used experts in mixture-of-experts models.

- Status: reported (One outlet, not independently verified yet.)
- Topic: Open Source
- Published: 2026-11-14T08:31:51+00:00
- Updated: 2026-11-14T09:09:25.369816+00:00
- Web page: https://ai-news.ikangai.com/story/llama-cpp-reaches-1-0-with-expert-caching-for-moe-models-larger-than-gpu-memory/
- Seen in: Latent Space
- Change: capability (New capability)
- Applies to: llama.cpp, ggml

## What changed

llama.cpp and ggml reached version 1.0, with llama.cpp adding a cache for frequently used experts in mixture-of-experts models and removing 35 deprecated functions from its public header.

llama.cpp and ggml have both reached version 1.0. The llama.cpp release adds a way for users of mixture-of-experts models larger than their GPU memory to keep a cache of frequently used experts. The 1.0 label comes with a stability declaration and the removal of 35 deprecated functions from llama.cpp's public header. Hugging Face announced the v1.0.0 release for the ggml-org projects and said the goal is to keep broad platform and model-type coverage while keeping the runtime "robust and lean" for downstream integrations. SGLang also announced its own 1.0 at the SGLang Summit.

The caching feature targets a specific bottleneck. A mixture-of-experts model routes each token to a small subset of its expert networks, so only some of its weights are used at a time, but the full set still has to be stored and moved. When the model does not fit in GPU memory, experts that are not resident must be loaded before they can run. A cache that holds the experts a workload selects most often avoids repeated loading and dispatch overhead, provided the routing actually repeats. A commenter in a LocalLLaMA discussion of the release argued that this advantage may shrink as models become sparser, because expert selections become less concentrated. The same thread included a question about whether MoE streaming methods used in other projects could be added to upstream llama.cpp, on the argument that many performance-focused forks make deployment choices harder. The announcement coverage does not include speed or memory figures for the cache, so how much it helps on a given model and hardware is not established.

Maintenance was the other concern raised in the discussion. A commenter pointed out that llama.cpp has "a ton of open issues and PRs" and asked for more maintainers. Hugging Face also announced Llama desktop apps built on llama.cpp, which it said are meant to cut the setup of running local models down to a few clicks.

## What you can do

Teams that run local MoE models could test the 1.0 expert cache on their own hardware.

---

AI wrote this summary from the sources listed, and nobody reviewed it individually before publication. Check the original source before you rely on it.

Cite as: IKANGAI AI News, "llama.cpp reaches 1.0 with expert caching for MoE models larger than GPU memory", 2026-11-14, https://ai-news.ikangai.com/story/llama-cpp-reaches-1-0-with-expert-caching-for-moe-models-larger-than-gpu-memory/
