Mastering Local Voice Control: Integrating Rhasspy with Home Assistant via MQTT

Are you seeking a voice assistant that respects your privacy and offers unparalleled customization? Mainstream voice platforms mean sending your commands to the cloud, limiting control and raising privacy concerns. For Home Assistant enthusiasts, Rhasspy provides a robust, fully offline, and open-source solution. Integrating Rhasspy with Home Assistant via MQTT empowers you to create a secure, cloud-independent smart home voice control system.

This guide dives deep into setting up Rhasspy, configuring custom voice commands, and linking it to your Home Assistant instance. We'll provide actionable steps, essential YAML configurations, and troubleshooting tips to give you complete mastery over your local voice commands.

Step 1: Installing Rhasspy via Docker

Rhasspy can run on various Linux systems, with Raspberry Pi (3B+ or newer) being a popular choice. Docker simplifies installation and management. Ensure Docker and Docker Compose are installed, then create a docker-compose.yaml file:

version: '3.8'
services:
  rhasspy:
    container_name: rhasspy
    image: 'rhasspy/rhasspy:2.5.11' # Use latest stable
    ports: ['12101:12101']
    volumes:
      - './profiles:/profiles'
      - '/etc/localtime:/etc/localtime:ro'
    devices: ['/dev/snd:/dev/snd'] # Access audio hardware
    restart: unless-stopped
    command: --user-profiles /profiles --profile en # Change 'en' to your language

Save and run docker-compose up -d. Access the Rhasspy web interface at http://[your_rhasspy_ip]:12101.

Step 2: Basic Rhasspy Configuration

Upon first access, select your language profile (e.g., 'en'). In the Rhasspy web UI:

• Audio Settings: Configure "Audio Recording" and "Audio Playing" to use your microphone and speaker. Check system ALSA settings if devices aren't visible.
• Voice Models: Choose resource-friendly models for Raspberry Pi: Porcupine (Wake Word), Pocketsphinx (STT), Fst (NLU), and PicoTTS (TTS). Set your desired wake word (e.g., "Hey Assistant").
• MQTT Setup: Enable and configure MQTT to connect to your Home Assistant's broker. Provide host, port (1883), and credentials if applicable.

Step 3: Integrating Intents with Home Assistant

Home Assistant listens for Rhasspy intents via its intent integration and MQTT.

3.1 Home Assistant Configuration

Add to your configuration.yaml:

# configuration.yaml
intent:

mqtt: # Ensure MQTT is also configured
  broker: your_mqtt_broker_ip
  port: 1883
  username: your_mqtt_username # If required
  password: your_mqtt_password # If required

Restart Home Assistant.

3.2 Creating a Custom Intent

In Rhasspy's "Sentences" tab, define an intent. For example, to control a light:

# Rhasspy Sentences (sentences.ini)
[ToggleLight]
(turn on|turn off){state} the (bedroom|living room){room} light

Click "Train" in Rhasspy. Now, create a Home Assistant automation:

# automations.yaml
- alias: "Rhasspy Toggle Room Light"
  trigger:
    platform: event
    event_type: rhasspy_ToggleLight # Matches [ToggleLight]
  action:
    - service: "light.turn_{{ trigger.event.data.state }}"
      data_template:
        entity_id: "light.{{ trigger.event.data.room }}_light"
    - service: mqtt.publish # Optional: provide voice feedback
      data_template:
        topic: 'hermes/dialogueManager/say'
        payload: >
          {
            "text": "Okay, {{ trigger.event.data.state }}ing the {{ trigger.event.data.room }} light.",
            "siteId": "{{ trigger.event.data.siteId }}",
            "sessionId": "{{ trigger.event.data.sessionId }}"
          }

This automation uses variables captured by Rhasspy (state, room) to control lights and provides a spoken confirmation.

Troubleshooting Common Rhasspy Issues

• Wake Word/STT Problems: Verify microphone selection (Rhasspy UI & OS). Adjust microphone gain. Reduce background noise. Test different wake word models or retrain.
• Intents Not Triggering: Check Rhasspy's MQTT connection (logs, MQTT Explorer on hermes/intent/#). Ensure Rhasspy is trained. Verify event_type in HA automation matches rhasspy_YOUR_INTENT_NAME exactly. Test sentences in Rhasspy's web UI.
• Rhasspy Web UI Unreachable: Confirm Docker container is running (docker ps). Check for port conflicts (12101). Verify firewall rules.

Advanced Configuration: Scaling & Customization

• Voice Feedback: Always include mqtt.publish actions in your HA automations to provide spoken responses from Rhasspy. Use siteId and sessionId from the trigger data to route responses correctly.
• Multiple Satellites: For whole-home coverage, deploy multiple Rhasspy instances (satellites) in different rooms. Each satellite connects to your central MQTT broker and routes intents. Ensure each satellite has a unique siteId (configured via --siteId bedroom in Docker command or env var).
• Custom Pronunciations: If Rhasspy mishears specific words (e.g., device names), add phonetic pronunciations in Rhasspy's "Pronunciations" tab (e.g., my_device M AY D IH V AY S), then retrain.

Real-World Example: Dynamic Scene Activation

Here’s an intent to activate a specific scene based on context:

# Rhasspy Sentences (sentences.ini)
[ActivateScene]
set the (mood|lighting|scene){scene_type} to (romantic|movie|reading){scene_name}

# automations.yaml
- alias: "Rhasspy Activate Scene"
  trigger:
    platform: event
    event_type: rhasspy_ActivateScene
  action:
    - service: scene.turn_on
      data_template:
        entity_id: "scene.{{ trigger.event.data.scene_name }}"
    - service: mqtt.publish
      data_template:
        topic: 'hermes/dialogueManager/say'
        payload: >
          {
            "text": "Activating {{ trigger.event.data.scene_name }} scene.",
            "siteId": "{{ trigger.event.data.siteId }}",
            "sessionId": "{{ trigger.event.data.sessionId }}"
          }

This allows for intuitive, context-aware control of complex lighting or ambiance settings with a simple voice command.

Best Practices for a Robust Local Voice System

• Prioritize Privacy: Keep all Rhasspy components local. Verify that no cloud services are accidentally enabled.
• Hardware Matching: Choose hardware appropriate for your Rhasspy models. For a central server, consider a more powerful mini-PC.
• Backup & Version Control: Regularly back up your Rhasspy profiles directory. For complex setups, consider version controlling your sentences.ini and Home Assistant automations.
• Iterative Development: Start with basic intents, test thoroughly, and incrementally add complexity.
• MQTT Security: While local, ensure your MQTT broker is password-protected. For external access, use TLS.

By mastering Rhasspy integration, you gain a powerful, private, and fully customizable voice interface for your Home Assistant smart home. Embrace the freedom from cloud reliance and build the intelligent environment you truly control.