NVIDIA released Nemotron 3.5 Bolt E4B 4GB on October 23 as a downloadable, locally served language model for conversational agents. Its model card specifies text input and text output, a Q4_K_M checkpoint, and a context limit of up to 64,000 tokens. The company positions it for voice assistants and devices, but an application still needs separate components to hear speech, produce audio, and connect the model to tools.

The big change

  • What changed: NVIDIA has released a quantized version of its Gemma 4 E4B based conversational model with local serving instructions and an optional draft model for faster generation. The checkpoint handles text, including the text exchanged with tools.
  • Why it matters: Developers designing a local assistant can now assess a released artifact and a documented runtime configuration instead of relying on an announcement alone. NVIDIA's memory and latency figures apply to measured hardware and workloads; the complete assistant needs its own budget.
  • What to watch: The published 6.18 GB model file and NVIDIA's roughly 4 GB runtime examples describe different things. Hardware, context length, optional drafting and any speech or vision components determine what a finished system requires.

What the release contains

Bolt E4B is NVIDIA's post-trained version of Google's Gemma 4 E4B IT. The model card lists 8 billion total parameters and 4 billion effective parameters. The “E4B” label refers to that effective count, while “4GB” appears in the product name and NVIDIA's discussion of target runtime workloads. The released GGUF weights use Q4_K_M quantization; a separate BF16 checkpoint is also available.

The Hugging Face file listing shows the Q4_K_M GGUF at 6.18 GB. NVIDIA's launch account says the model can use roughly 4 GB in target workloads. Its measured peak GPU memory ranges from 3.1 GB to 5.0 GB across the RTX and Jetson configurations it publishes. Those figures are workload and hardware measurements, not the file size or a guarantee that the whole application fits in 4 GB. The blog reports, for example, 3.5 GB of peak GPU memory and 548 milliseconds to first token on an RTX 5070 with no cached context, about 3,000 input tokens and 50 output tokens. With 32,000 cached tokens, 2,000 new input tokens and 256 output tokens on the same card, it reports 4.3 GB and 1,238 milliseconds. Both are NVIDIA measurements at batch size one using Q4_K_M, llama.cpp with CUDA and AIPerf.

The model card's quick start includes a pinned CUDA llama.cpp container and configurations with or without a second checkpoint called DFlash. The card's summary still says “llama.cpp (recipe pending),” although the detailed quick start supplies commands. That inconsistency is in the published documentation; the runnable instructions are present, but BIG CHANGE has not executed them. DFlash is an optional speculative decoding drafter: it proposes tokens that Bolt verifies. Its separate model card lists a 737 MB GGUF file and says it must be loaded with the target checkpoint. That extra file and its runtime memory belong in any assisted serving plan.

Text model, broader assistant

NVIDIA's “voice assistant” use case describes an application role, not a speech input or output in this checkpoint. The model card lists text on both sides. NVIDIA's Reachy Mini example pairs Bolt with Cosmos 3 Edge for visual understanding and scripts for robot movement. A spoken assistant would likewise need speech recognition and synthesis around the text model. Bolt can use tool calls and a conversational mode intended to make replies shorter and more suitable for speech, but those features do not supply microphones, audio generation or device actions on their own.

This also makes Bolt a distinct release from NVIDIA's earlier Nemotron 3.5 Super VL vision-language weights and Nemotron 3 Diarization audio model. The models share a family name but serve different modalities and tasks.

What the speed and quality results show

NVIDIA reports roughly 40% higher average scores than its previous Nano 4B model across its selected conversational-agent benchmarks, and up to 80% fewer output tokens than compared peers in reasoning-enabled settings. These are vendor-run comparisons, not independent results or a universal speed advantage. Its model-card table separates BF16 from Q4_K_M scores and says most tests used reasoning mode. Results vary by task: in the BF16 table Bolt scores 55.10 on Tau2-Bench against 75.06 for Qwen 3.5 4B, while on RoleBench it scores 67.05 against Qwen's 53.66. Neither pair describes the performance of every local assistant.

The launch blog's response-time table is more useful for sizing the stated workload than for promising a user experience. It reports time to first token, throughput and peak GPU memory for specific cards, context and output lengths. It does not measure the full path from a person's speech through transcription, tool execution and spoken reply. Teams evaluating the release still need to measure that end-to-end path on their own target hardware and application data. NVIDIA recommends that application-specific testing in the model card.

The weights are available on Hugging Face under the OpenMDW 1.1 license, and the card says commercial and noncommercial use is permitted. The card also points to Apache 2.0 terms for the Gemma 4 base. Its linked download and local serving recipes document access, while they establish no all-in hardware cost or paid service price for a deployed assistant.

Sources & further reading