NVIDIA Shrinks a Voice Assistant's Brain Enough to Keep Your Words at Home
NVIDIA's new open model is small enough to run on a home PC or a small device board, which means what you say to an assistant may not have to leave the room.
The most private voice assistant is the one that never sends your words anywhere. NVIDIA just made that easier to build.
On 23 October, NVIDIA released Nemotron 3.5 Bolt E4B, an open model built on Google's Gemma 4. It's designed to power voice assistants, game characters and robots, and it can run in roughly 4GB of memory. That's small enough for an ordinary gaming PC or one of NVIDIA's small Jetson boards. In practice, your request can be understood and answered on the device in front of you, without a trip to someone else's server.
Why size matters for privacy
Most assistants today send what you say to the cloud, because the models that understand you are too big to run at home. NVIDIA says privacy concerns are one of the reasons companies hold back from putting large models into products. A model this small changes the trade. A car, a smart speaker or a factory screen could keep conversations local and still answer quickly.
NVIDIA claims Bolt scores about 40% higher than its previous small model on conversation and tool use tests, and gives answers up to 80% shorter. These are NVIDIA's own benchmark results and haven't been independently tested. Some of the tests NVIDIA used haven't been published yet either.
What builders get
The model has 8 billion parameters in total, but NVIDIA says it works like a 4 billion parameter model, which is how it fits in about 4GB when compressed. It can remember long conversations, up to 64,000 tokens, and NVIDIA says it supports 35 languages.
Two switches matter for voice products. A "conversational mode" pushes it toward short replies that sound natural when spoken aloud. A reasoning setting lets developers turn step by step thinking on for hard requests and off when speed matters most. The model can also call other apps to carry out tasks across a conversation, such as changing a setting or looking something up.
NVIDIA's model card says it is ready for commercial use under the OpenMDW license. NVIDIA also published a full-precision version for companies that want to retrain it on their own data.
What it doesn't solve
Bolt only handles text. A full voice assistant still needs separate speech tools to listen and talk, and each of those pieces has to stay on the device for the privacy promise to hold. Running locally also says nothing about what an app maker chooses to log or upload later.
There is also a hardware angle. NVIDIA names its own chips, from GeForce RTX graphics cards to Jetson boards, as the targets. The model is open, but it is also a reason to buy NVIDIA hardware.
Still, the direction is clear. As capable models get smaller, "it has to go to the cloud" is becoming a weaker excuse. What to watch: independent tests of NVIDIA's claims, and whether device makers ship fully local assistants built on it.
Sources
- Introducing NVIDIA Nemotron 3.5 Bolt E4B 4GB (NVIDIA Research)
- NVIDIA-Nemotron-3.5-Bolt-E4B-4GB model card (Hugging Face)
- NVIDIA-Nemotron-3.5-Bolt-E4B-BF16 model card (Hugging Face)
Photo: "NVIDIA Jetson Nano Developer Kit" by SparkFun Electronics, via Wikimedia Commons.jpg), CC BY 2.0.