Home Assistant 2026.8 landed on August 5 with the tagline “Approachable by design,” and the headlines went to the friendly stuff: new installs answer on port 80 instead of :8123, “Developer Tools” is now just “Tools,” and around 43 “advanced” labels got quietly deleted from the interface. All fine. But scroll down to the new-integrations list and there it is, sitting between Lyngdorf amplifiers and pool controllers: a native llama.cpp integration. Home Assistant can now use a local language model as a conversation agent, out of the box, with no cloud account anywhere in the loop.
For a house that runs everything on a used HP EliteDesk 800 G4 and treats cloud dependencies as a bug, this is the most interesting line in the entire release post. So I spent the weekend finding out what it actually means when your AI brain is a CPU-only mini PC that also has a day job.
What actually shipped
Two things, really. The llama.cpp integration (contributed by allenporter, who also maintains a good chunk of HA’s AI plumbing) lets you point Home Assistant at a local llama.cpp server — or any endpoint that speaks the OpenAI-compatible chat completions API — and use it as a conversation agent for Assist. You can set up several agents, each with its own model and its own instructions. Alongside it came a LiteLLM integration, which does the same trick through a proxy that can front almost any model provider, hosted or self-hosted.
The important nuance: Home Assistant is not running the model itself. There’s no bundled inference engine, no add-on that magically eats your RAM. You bring your own llama.cpp server, and HA talks to it over HTTP. That’s the right call, in my opinion — inference is exactly the kind of workload you want to be able to place, throttle, and kill independently of the process that unlocks your front door.
The setup, honestly documented
My HAOS box is an EliteDesk 800 G4 mini with an i5-8500T and no GPU worth mentioning. Since HAOS is deliberately not a general-purpose Linux box, the llama.cpp server has to live somewhere else — in my case a second machine on the LAN, though a container on the same host works too if you run Supervised or Core. llama.cpp’s llama-server gives you the OpenAI-compatible endpoint with one command and a GGUF model file.
On the Home Assistant side, the flow is what you’d expect from a 2026-era integration: add it from the UI, give it the server URL — and note that the integration wants the full path including protocol, port, and the /v1 suffix, which is the one place I fumbled — then pick your options. You can write the agent’s instructions using Home Assistant templating, choose whether the model can control entities or only answer questions about them, and either accept recommended sampling defaults or set max tokens, temperature, and top-p yourself. Crucially, the model only ever sees entities you’ve explicitly exposed to Assist. That boundary matters more with LLMs than it ever did with intent matching, because a language model will cheerfully improvise with whatever context you hand it.
What a CPU-only box can honestly do
Here’s the part the release notes won’t tell you, because it depends entirely on your hardware. On an 8th-gen desktop CPU with no accelerator, small quantized models — the 3B-parameter class at Q4 quantization — are the realistic ceiling. They fit in a few gigabytes of RAM and produce answers at a pace I’d describe as “thoughtful colleague,” not “instant assistant.” Single-digit tokens per second, in my experience. A 7B or 8B model runs, technically, the way a loaded moving van technically goes uphill.
Whether that’s acceptable depends on what you ask of it. For the queries where an LLM genuinely earns its keep — “which windows are still open?”, “summarize what happened overnight” — waiting a few seconds for an answer that’s actually synthesized from my home’s state feels fair. For “turn off the kitchen lights,” it’s absurd, and you shouldn’t use it for that anyway: Assist’s built-in intent engine handles the direct commands instantly and deterministically. The pattern that works is local intents first, LLM as fallback — which Assist has supported for a while and which turns the slow model into a specialist you consult, not a bottleneck you live behind.
The same release also ships GPT-5.6 support. That’s the point.
I want to be fair here: 2026.8 also updated the OpenAI Conversation integration with GPT-5.6 support, and the Anthropic integration just reached platinum on the quality scale. The cloud models are better at this. Dramatically better. If you want an assistant that understands “make it cozy in here” with all its implied ambiguity, a frontier model through an API will outclass anything my EliteDesk’s neighbor can serve.
But that’s always been the trade, and what 2026.8 changes is that the local end of it is now a first-party, UI-configured choice rather than a custom-integration adventure. The community has been wiring llama.cpp into HA for years through projects like Home LLM and various OpenAI-shim workarounds. Those deserve credit — they proved the demand. Having it in core, with entity exposure controls and proper config flow, is the difference between “possible for people who read forums” and “possible.”
Should you bother?
If you have any spare box with decent RAM — or better, anything with a GPU or a recent Mac — yes, try it. The marginal cost is an afternoon. If your entire smart home is one Raspberry Pi, no: the integration needs an external inference server, and that’s not your Pi’s fight.
My own verdict after a weekend: it stays, as a fallback agent with a small model, answering the questions that intent matching can’t. It’s slow, occasionally confidently wrong about which sensor is which, and entirely mine. Every query, every response, every half-second of my house’s state stays on hardware I can unplug. In a release themed around approachability, the most on-brand feature isn’t the missing port number — it’s that running your own AI stopped requiring anyone’s permission.
Primary sources: the Home Assistant 2026.8 release notes, the llama.cpp integration docs, and the llama.cpp project.
Leave a Reply