February 8, 2025

TL;DR: It’s our objective for 2023 to let customers management Residence Assistant in their very own language. Mike Hansen, creator of Rhasspy, has joined Nabu Casa to guide this effort. We’re beginning off by constructing a collection of intent matching sentences in every language.

Normally, the month of December is supposed to mirror again. Nevertheless, we already did that final month after we hosted the State of the Open Home 2022. We didn’t solely mirror, we additionally introduced our focus for subsequent 12 months: 2023 goes to be the 12 months of voice.

It’s our objective for 2023 to let customers management Residence Assistant in their very own language.

It’s a giant and daring objective, however achievable given the precise constraints. The quantity of labor laid out for us will be summarised as follows:

Spoken languages times possible actions

Our #1 precedence is supporting completely different languages. There are sufficient initiatives on the market making an attempt to create an English voice assistant. However for us, that simply doesn’t minimize it. Individuals want to have the ability to converse in their very own language, as that’s the most accessible and solely acceptable language for a voice assistant for the good residence.

To maintain the quantity of labor forward of us manageable, we’re going to restrict the variety of attainable actions and deal with the fundamentals of interacting along with your good residence. No net searches, making calls, or voice video games. And undoubtedly no “by the best way”s!

We’re going to begin with a number of actions and construct up the language fashions round that. Residence Assistant helps 62 completely different languages in its person interface. And it’s our objective to assist all these languages with voice. We predict that we are able to obtain that by leveraging Residence Assistant’s strongest asset: our neighborhood.

Our history with voice assistants

If you follow the news, it might sound like voice assistants have failed. Amazon is set to lose $10 billion on Alexa this year and is planning layoffs. Google too, is decreasing its support for Google Assistant because it’s making an attempt to chop prices. The reality is that voice, as the subsequent computing platform that drives billions of {dollars} of additional income, has failed. As an alternative, customers primarily use their voice assistants to handle buying lists, set timers, play music, and management their properties. Voice has failed being a income, it has not failed its customers.

With Residence Assistant we’ve all the time been excited about voice. We used to work with Snips again within the day, however they acquired acquired and shut down. We labored with Stanford on their Almond/Genie platform, however it’s a analysis pushed challenge that by no means acquired manufacturing prepared. And sure, you should utilize Residence Assistant to ship all of your knowledge to the clouds of Google and Amazon to leverage their voice assistants, however you shouldn’t have to surrender your privateness to activate the lights by voice.

Probably the most promising challenge out there’s Rhasspy, created by Mike Hansen. A challenge that permits individuals to construct their very own native voice assistant, which might additionally tie into Residence Assistant. Rhasspy stands out from different open supply voice initiatives as a result of Mike doesn’t deal with simply English. As an alternative, his objective is to make it work for everybody. That is going nice as Rhasspy helps already 16 completely different languages at this time.

With Residence Assistant we wish to make a privateness and domestically centered good residence out there to everybody. Mike’s method with Rhasspy aligns with Residence Assistant, and so we’re comfortable to announce that Mike has joined Nabu Casa to work full-time on voice in Residence Assistant.

Iterating in the open

With Home Assistant we prefer to get the things we’re building in the user’s hands as early as possible. Even basic functionality allows users to find things that work and don’t work, allowing us to address the direction if needed.

A voice assistant has a lot of different parts: hot word detection, speech to text, intent recognition, intent execution, text to speech. Making each work in every language is a lot of work. The most important part is the intent recognition and intent execution. We need to be able to understand your commands and execute them.

We started gathering these command sentences in our new intents repository. It can quickly energy the present dialog integration in Residence Assistant, permitting you to make use of our app to jot down and say instructions.

The dialog integration is uncovered in Residence Assistant through a service name and can be out there via an API to external applications or scripts. This enables builders to experiment with sending instructions from varied sources, like a telegram chatbot.

How you can help

For each language we’re collecting sentences of commands that control your smart home in our intents repository. Every sentence will must be annotated with its intention.

Take for instance the sentence: Activate the bed room lights. Write it up like Activate the realm lights and it turns into a generic command to activate all of the lights in a particular space. Now we have to acquire all the opposite variations too.

We’ve created a YAML-based format to declare and test these sentences. The following step is that we’d like you 🫵

For every language we’re going to want a number of language leaders. Language leaders are answerable for reviewing the contributions of their language and ensuring that they’re grammatically appropriate. If you wish to apply to be a language chief, be part of us in #devs_voice on Discord or open a difficulty in our intents repository.

We additionally want people who wish to contribute sentences to their language to assist construct out our assortment. See our intents repository on how to get started.