Voice Recognition in AEM Screens
Important Privacy Information
When using the voice recognition feature follow all applicable legal and ethical guidelines for your region (including but not limited to providing a visible notice to end users that the player is using Voice Recognition). Adobe Inc., does not receive, store or process any of the voice related information. The AEM Screens players use the standard web speech API built into the browsing engine. Behind the scenes this API sends a wave form of your speech to Google's servers for conversion from speech to text and this text is matched by the player against configured keywords.
Refer to Google Privacy White-paper on web speech API for more details.
The voice recognition feature allows content change in an AEM Screens channel driven by voice interaction.
A content author can configure a display to be voice enabled. The purpose of this feature is to allow customers to utilize speech as a method of interacting with their displays. Some similar use cases include finding product recommendations in stores, ordering menu items at diners and restaurants. This feature increases accessibility for users and can greatly enhance customer experience.
The player hardware must support voice input, such as a microphone.
Implementing Voice Recognition
The voice recognition feature is available only on Chrome OS and Windows players.
To implement voice recognition in your AEM Screens project, you must enable the voice recognition for the Display and associate each channel with a unique tag to trigger a channel transition.
The following section describes how you can enable and use the voice recognition feature in an AEM Screens project.
Viewing Content in Full Screen or Split Screen Channel Switch
Before you use voice recognition feature, make sure you have a project and a channel with content set up for your project.
- The following example showcases a demo project named VoiceDemo and three sequence channels Main , ColdDrinks , and HotDrinks , as shown in the figure below.To learn how to create a channel or add content to a channel, refer to Creating and Managing ChannelsOr,You can create three sequence channels Main , ColdDrinks , and HotDrinks , and one additional 1x2 Split Screens channel SplitScreen as shown in the figure below.
- Navigate to each of the channel and add content. For example, navigate to VoiceDemo --> Channels --> Main and select the channel. Click Edit from the action bar to open the editor and add content (images/videos) as per your requirement. Similarly, add content to both ColdDrinks and the HotDrinks channel.The channels now contain assets (images), as shown in the figures below.Main :ColdDrinks :HotDrinks :If you have added Split Screens channel to your project, navigate to SplitScreen and drag and drop two embedded sequences and add paths to both the ColdDrinks and HotDrinks channel as shown in the figure below.
Assigning Channel to a Display and Enabling Voice Recognition
- Create a display in the Locations folder, as shown in the figure below.
- Assign the channels Main , ColdDrinks , and HotDrinks to your LobbyDisplay . Additionally, if you are using the SplitScreen channel for your project, make sure you assign that too to the display.If you have created a split screen channel, assign the SplitScreen channel too to your display.
- Set the following properties to each of the channel, while assigning the channel.Channel NamePrioritySupported EventsMain2Initial Load, Idle Screen, TimerHotDrinks1User InteractionColdDrinks1User InteractionSplitScreen1User Interaction
- Once you have assigned channels to a display, navigate to the LobbyDisplay and select the display. Select Properties from the action bar.
- Navigate to the Display tab and enable Voice enabled option under Content .It is mandatory to enable the voice recognition feature from the display.
Viewing the Content in the Chrome Player
Once the preceding steps are complete, you can register your chrome device to view the output.
Refer to Device Registration to learn how to register a device on an AEM Screens player.
Desired Output for Sequence Channel
The Main channel is playing its content, but when you use words with keyword hot such as I would like to have a hot drink , the channel starts playing the contents of the HotDrinks channel.
Similarly, if you use word with a keyword cold such as I would like to haves something cold , the channel starts playing the contents of the ColdDrinks channel.
Desired Output for Split Screens Channel
The Main channel is playing its content, but when you use words with keyword hot and cold together such as I would like to see the menu for hot and cold beverages , the channel starts playing the contents of the SplitScreen channel. If you say back to main menu , it switches back to the main channel.