# NVIDIA's Nemotron 3.5 Bolt targets local agents within a 24GB memory budget

> NVIDIA's Nemotron 3.5 Bolt is a 74B-total, 9B-active vision-language model designed to run on a 24GB memory budget with tool calling.

- Status: reported (One outlet, not independently verified yet.)
- Topic: Models
- Published: 2026-11-09T13:03:45+00:00
- Updated: 2026-11-09T13:11:34.635012+00:00
- Web page: https://ai-news.ikangai.com/story/nvidia-s-nemotron-3-5-bolt-targets-local-agents-within-a-24gb-memory-budget/
- Seen in: AI Engineering
- Change: capability (New capability)
- Applies to: Nemotron 3.5 Bolt, Nemotron 3.5 Super VL, Unsloth Desktop, llama.cpp, IQ2_XXS

## What changed

NVIDIA's Nemotron 3.5 Bolt is a new 74B-total, 9B-active vision-language model that runs on a 24GB memory budget with tool calling, derived from Nemotron 3.5 Super VL.

NVIDIA's Nemotron 3.5 Bolt is a vision-language model with 74B total parameters and 9B active per token. It is built to run on a 24GB memory budget, accepts text, images and video, and supports tool calling. Unsloth has published a downloadable GGUF build, IQ2_XXS, that is 22GB and loads in Unsloth Desktop or llama.cpp. Local use means model inference does not have to go to a hosted API.

The model starts from Nemotron 3.5 Super VL, a 120B-parameter model. NVIDIA reduced the architecture to 74B parameters, applied additional training, and then compressed the weights. The compression is not uniform. Expert weights are stored at roughly two bits each. Mamba linear layers use NVFP4, while attention, embeddings and other selected components stay at higher precision. NVIDIA says quantization-aware distillation recovers accuracy lost in compression. The 9B active count is a separate saving: each token passes through selected parts of the network, which reduces computation per token. Quantization reduces the memory needed to hold the full model.

The 24GB figure needs some context. The 22GB download is the model weights, not the full running application. Unsloth lists 24 to 32GB for the IQ2_XXS build and recommends budgeting around 32GB for short-context use. Images, video and longer conversations need more memory. Unsloth also enables a LiLiCorr draft model by default to speed up generation, which adds roughly 1 to 2GB. By that accounting, 24GB is the lower end of what the build needs, and a 32GB machine leaves more room for the runtime.

The Unsloth guide describes two ways to run it. In Unsloth Desktop, a user installs the application, searches for Nemotron 3.5 Bolt in Chat or Model Hub, selects the 74B-A9B model, downloads the IQ2_XXS build and starts with a short context rather than the full 128K-token window. The guide warns users to choose the full model name carefully, since a separate and smaller model called Bolt E4B also exists. For application developers, the guide gives a command that starts a local server with a 4,096-token context that handles one request at a time. The server is reachable at http://127.0.0.1:8001.

Bolt can generate tool calls, but the application running it has to execute them. Permissions, approval steps and execution limits remain the developer's responsibility in the agent runtime. NVIDIA reports approximately 84 output tokens per second with LiLiCorr on its RTX Spark test system. That system used 128GB of memory, so the figure should not be read as the speed of every 24GB setup.

## Why it matters

Teams building local agents can get visual understanding and tool calls without sending inference to a hosted API, but 24GB is the lower end of what the source describes.

## What you can do

Before choosing hardware for a local agent, test a full workflow on the target machine, including image processing, tool calls and error recovery, not just generation speed.

---

AI wrote this summary from the sources listed, and nobody reviewed it individually before publication. Check the original source before you rely on it.

Cite as: IKANGAI AI News, "NVIDIA's Nemotron 3.5 Bolt targets local agents within a 24GB memory budget", 2026-11-09, https://ai-news.ikangai.com/story/nvidia-s-nemotron-3-5-bolt-targets-local-agents-within-a-24gb-memory-budget/
