July 17, 2024
Yr of the Voice – Chapter 1: Help

This yr is Residence Assistant’s yr of the voice. It’s our purpose for 2023 to let customers management Residence Assistant in their very own language. In the present day, one month into 2023, we begin our first chapter.

At Residence Assistant we consider that expertise is supposed to be performed with, and tasks needs to be usable as quickly as doable. Along with the group we will then iterate and refine. That’s why immediately, we’re delivering a primary expertise for 22 languages to work together with Residence Assistant. Oh, and we’re additionally releasing some enjoyable stuff that we cooked up alongside the best way.

To observe the video presentation of this weblog submit, together with reside demos, verify the recording of our live stream.

Intentions

The core of a voice assistant is to be able to understand the intention of a spoken sentence. What is it the user wants to do? To extract these intentions we created our own template sentence matching format and intent recognizer named Hassil.

This new format is utilized by our new Home Assistant Intents mission. The purpose of this mission is to gather house automation sentences in each doable language. Because it’s begin a month in the past, now we have had 112 folks contribute. The mission now helps 22 languages and 14 more are in progress.

Assist

We have added a new feature to Home Assistant: Assist icon Assist. It allow users to use natural language to control Home Assistant. It is powered by Hassil and the sentences from the Home Assistant Intent project.

We want Assist to be as accessible to as many people as possible. To do this, we made it work without requiring extra hardware – just update to Home Assistant 2023.2 and you can start! Through a combination of smart algorithms combined with sheer brute force (we are collecting a lot of sentences), we have been able to make a system that works for most common sentences. Support for more powerful, AI-powered, intent recognizers might come in the future as an opt-in feature.

Assist is enabled by default in the Home Assistant 2023.2 release. Tap the new Assist icon Assist icon at the top right of the dashboard to use it.

Assist documentation.

Screenshot of the Assist dialog

Assist on Android Wear

We want to make it as easy as possible to use Assist. To enable this for Android users, we have added a new tile to the Android Wear app. A simple swipe from the clock face will show the assist button and allows you to send voice commands.

Assist on Android Wear documentation.

The brand new tile is at the moment accessible within the Android beta and can be a part of the following Android launch.

Assist via Siri and Apple Shortcuts

For Apple devices we have been able to create a fully hands-free experience by integrating with Siri. This is powered by a new Apple Shortcut action called Assist, which is part of the Home Assistant app. This shortcut action can also be manually triggered from your Mac taskbar, iPhone home screen or Apple Watch complication. We have two ready-made shortcuts that users can import from the documentation with a single tap to unlock these features.

Assist via Siri and Apple Shortcuts documentation.

The Help shortcut can be accessible within the Mac and iOS beta channel immediately and can be a part of the following launch for iOS and Mac.

Customized Sentences

With Residence Assistant we consider that each house is uniquely yours and that technology should adapt to you, not the other way around. That’s why we have architected Home Assistant to allow users to extensively customize their experience. Our Assist feature is no different.

  • Are you into Game of Thrones and want every response to be “hodor”?
  • Want to turn on lights in rooms by saying “Hocus pocus living room”?
  • Want to trigger your party mode script using a custom sentence?

Assist includes support for custom sentences, responses and intents, allowing you to achieve all of the above, and more. We’ve designed the custom sentence format in a way that it can be easily shared with the community.

Read the documentation on get began.

In a future launch we’re planning on including a consumer interface to customise and import sentences.

Custom Assist engines

By default Assist is powered by our own intent recognizer. It is local but it’s limited to controlling devices. Maybe you want to be able to ask more wide-range queries or you are looking for a conversational AI that will make up responses and present it as the truth. For such cases the Assist feature supports swapping out its engine that handles all Help interactions.

The Residence Assistant 2023.2 launch contains two various Help engines that you would be able to allow: Google Assistant and OpenAI GPT-3.

The Google Assistant Help engine is ready to management your units you probably have linked up your Residence Assistant occasion to Google Assistant.

All methods to intereact with Help will work, as they don’t seem to be certain to the the Help engine that’s getting used. So if you happen to ever needed to make use of Google Assistant in your HomePod, now you possibly can 🤭

The OpenAI GPT-3 Help engine will course of all of your interactions utilizing GPT-3, a sibling of the notorious ChatGPT. It isn’t capable of management your own home or provide help to automate your own home. Something you ask it might or is probably not factually right. However it may be enjoyable!

In a future launch we’re planning to make it doable to configure a number of Help engines to deal with interactions.

What’s next

For Year of the Voice – Chapter 1 we focused on building intent recognition into Home Assistant while relying on Google and Apple to do the hard parts (speech recognition). This allowed us the fastest path to get something to the community to play with.

We will continue collecting home automation sentences for all languages (anyone can help!). Updates can be included with each main launch of Residence Assistant.

Our subsequent step is integrating Speech-to-Textual content and Textual content-to-Speech with Help. We don’t have a timeline but when that can be prepared. Keep tuned!

Credits

A lot of people have worked very hard to make all of the above possible.

Technology:
Mike Hansen, Paulus Schoutsen, Daniel Shokouhi, Zac West, Rosemary Orchard, Tronikos

Language Leaders:
@AalianKhan, @Ahmed-farag36, @alpdmrel, @arunshekher, @auanasgheps, @benjaminlecouteux, @bluefoxlee, @cibernox, @cvladan, @davefx, @dinhchinh82, @dsimop, @duhow, @easterapps, @ErnestStaug, @fadamsen, @flexy2dd, @gabimarchidan, @haim-b, @halecivo, @HepoH3, @hertzg, @hristo-atanasov, @huusissa, @joaorgoncalves, @larsdunemark, @leranp, @LubosKadasi, @makstech, @mojikosu, @MTrab, @nagyrobi, @schizza, @Scorpoon, @skynetua, @spuljko, @tetele, @TheFes, @Uriziel01, @xraver, @zubir2k

Voice Group:
@Alexivia, @Atalonica, @AwesomeGuy000, @BossNeo, @CedricFinance, @Davidsoff, @EmilZackrisson, @FragMenthor, @InfiniteBed, @Kalma-House, @Licmeth, @Marlo461, @N3rdix, @Nismonx, @Robin-St, @TaQuangTien, @ThomDietrich, @TomaszPilch, @Wojciechgc, @alessandroias, @bemble, @berendhaan, @dejan2101, @dependabot[@bot], @dobromir-hristov, @frenck, @hugovsky, @iddiek, @jfisbein, @jharrvis, @jorclaret, @kamildoleglo, @kblin, @khymmera, @kroimon, @lellky, @ludeeus, @lukahra, @lunmay, @mardito, @martindybal, @mib1185, @michaelmior, @orrc, @pckahrs, @piitaya, @pmentis, @poltalashka, @rPonuganti, @rechin304, @relust, @rickydg, @rpochot, @rrakso, @rumbu13, @sanyatuning, @tasmin, @thecode, @waltlillyman, @witold-gren, @x15pa3ck15x, @yuvalabou