A local Home Assistant voice assistant: setup and privacy

When you say “turn off the kitchen lights,” the system has to hear the phrase, turn audio into text, identify the intent, and perform the action. If you want a spoken reply, another step converts text back into speech. In Home Assistant, these parts form an Assist pipeline: speech-to-text, a conversation agent, and text-to-speech. Wake-word detection can be handled separately on compatible hardware.

These separate steps let you choose where data is processed. A microphone may be in a room, Home Assistant on a home server, and a speech engine on another computer on your local network. Check what runs locally at every step instead of assuming that installing Home Assistant alone keeps all audio on your network.

Home Assistant’s official guide recommends a local speech-to-text engine—Whisper or Speech-to-Phrase—and Piper for text-to-speech. After starting the components, add them through the Wyoming integration, then create an assistant under Settings → Voice assistants. Choose Home Assistant as the conversation agent if you want common control commands to work without an LLM.

Try Assist in the interface before buying a dedicated voice satellite. Check the language, room and device names, and whether short everyday phrases are understood correctly. Speech engines have different CPU, memory, and speed requirements; a small server can introduce noticeable delays, especially with general-purpose recognition. Speech-to-Phrase is optimized for a limited set of control phrases, while Whisper handles broader speech but needs more resources.

Once configured, temporarily disconnect the home network from the internet while keeping the local network and server running. Try several commands and listen to the responses. If recognition or speech stops working, one pipeline step may depend on a cloud service or downloading a model. Restore connectivity and inspect the engine selected for each step.

A microphone in a room raises a privacy question even when audio stays at home. Consider a physical mute button, a visible indicator, and placement outside bedrooms or other private areas. If you add a third-party agent or speech service, review its settings and audio handling policy: running other parts locally does not make an external service local.

Choose an easy-to-notice, reversible action for your first test: switching a light, changing brightness, or asking for the temperature. Give devices clear names and assign them to the right rooms. Try a few different phrasings and see what happens when the assistant does not understand.

Do not begin with voice control for locks, alarms, garage doors, or other actions with security consequences. Consider authentication, confirmation, and a backup control method; the available safeguards depend on the integration and hardware. A voice assistant should be a convenient interface, not the only barrier protecting critical functions.