Show Menu
TOPICS×

Implement Analytics for Digital Assistants

With recent advances in cloud computing, machine learning, and natural language processing, digital assistants are moving out of the dark ages of "clippy" and becoming part of everyday life. Consumers are starting to talk to their devices and expecting them to listen, understand, and respond in natural, human-like ways to phrases like, "Alexa, turn on the family room lights," or "Okay Google, What's the weather like outside?"
As these platforms become more established, brands can present their services to consumers in these same realistic and lifelike ways. For example, consumers can ask things like:
  • "Alexa, ask my car when it needs an oil change."
  • "Cortana, what is the balance of my checking account?"
  • "Siri, send John $20 for dinner last night from my banking app."
This white paper provides an overview of how best to use the Adobe Analytics Cloud to measure and optimize these types of experiences.

Digital Experience Architecture Overview

Most digital assistants today follow a similar high-level architecture:
  1. Device: There is a device (like an Amazon Echo or a phone) with a microphone that allows the user to ask a question.
  2. Digital assistant: That device interacts with the service that powers the digital assistant. This service is where a lot of the magic happens. It is where the speech is converted into machine understandable intents and the details of the request are parsed out. Once the user's intent is understood, the digital assistant passes the intent and details of the request to the app that handles the request.
  3. "App": The app can either be an app on the phone or a voice app. The app is responsible for responding to the request. It responds to the digital assistant and the digital assistant then responds to the user.

Where to implement Analytics

One of the best places to implement Analytics is in the app. The app receives the intent and the details about the intent from the digital assistant and decides how to respond.
There are two times during the life-cycle of a request that can be helpful to call the Analytics Cloud.
  1. When the request is sent to the "App." If you need additional context about the user before you respond to the request, you should leverage the Audience Manager capability to get the segments that they belong to.
  2. After the response is returned from the app. If you are just interested in recording what happened with the customer for future optimization, send a request to Adobe Analytics after the response has been returned. This way you have the full context of what the request was and how the system responded.

New Installs

For some of the digital assistants you will get a notification when someone installs the skill. This is especially the case when there is authentication involved. At this time you should send Adobe an Install event by setting the context data to a.InstallEvent=1 . Note that this is not available on all platforms but is helpful when it is present for looking at retention. The following code sample sends in Install , Install Date , and AppID .
Code Sample
GET 
/b/ss/[rsid]/0?vid=[UserID]&c.a.InstallEvent=1&c.a.InstallDate=2017-04-24&c.a.AppID=Spoofify1.0&c .OSType=Alexa&pageName=install 
HTTP/1.1 
Host:  
<xref href="https: sc.omtrdc.net="" " format="http"  scope="external">
  sc.omtrdc.net 
 Cache-Control: no-cache 
</xref href="https:>

Multiple Assistants or Multiple Apps

It is likely that you will develop apps for multiple platforms. The best practice is to include an app ID with each request. This can be set in the a.AppID context data. Follow the format of [AppName] [BundleVersion] , for example, BigMac for Alexa 1.2
Code Sample
GET /b/ss/[rsid]/0?vid=[UserID]&c.a.AppID=Spoofify1.0&c.a.Launches=1&c.Product=AmazonEcho&c.OSType=Alexa&pageName=install  HTTP/1.1 
Host: [prefix].sc.omtrdc.net 
Cache-Control: no-cache

 GET /b/ss/[rsid]/0?vid=[UserID]&c.a.AppID=Spoofify2.0&c.a.Launches=1&c.Product=GoogleHome&c.OSType=Android&pageName=install  HTTP/1.1 
Host: sc.omtrdc.net 
Cache-Control: no-cache

User/Visitor Identification

The Analytics Cloud uses the Identity Service (ECID) service to tie interactions across time to the same person. Most of the digital assistants will return a userID that you can use to keep the activity for different users separate in the ECID service. In most cases, this is what you should pass in as the ECID service. Some platforms return a userID that is longer than the 100 characters that the Analytics limit allows. If that is the case, Adobe recommends that you hash the userId to a fixed-length value using a standard hashing algorithm, such as MD5 or Sha1.
While you can use the ECID service for this, doing so provides value only if you are trying to map ECIDs across different devices (e.g. Web to Digital Assistant). If your app is a mobile app (e.g. a deep link) you should use the SDK as is and send the information along. The userID can be attached to the ECID service using the setCustomerID method, allowing for better device stitching. However, if your app is a service, use the userID provided by the service as the ECID, as well as setting it in the setCustomerID. This allows you to see how people are using the digital assistant over time.
Code Sample
GET /b/ss/[rsid]/0?vid=[UserID]&pageName=[intent]  HTTP/1.1 
Host: sc.omtrdc.net 
Cache-Control: no-cache

Sessions

Because digital assistants are conversational, they often have the concept of a session. For example:
Consumer: "Ok Google, call a cab for me"
Google: : "Sure, what time would you like?"
Consumer: "8:30pm"
Google: "Sounds good, the Driver will be by at 8:30pm"
These sessions are important to keep in context. They help collect more details and make the digital assistants more natural.
When implementing Analytics on a conversation, there are two things you should do when a new session is started:
  1. Reach out to Audience Manager: Get the relevant segments that a user is a part of so that you can customize the response. (For example, this person currently qualifies for the multi-channel discount.)
  2. Send in a new session or launch event: When you send the first response to Analytics, include a launch event. Usually, this can be sent by setting context data of a.LaunchEvent=1 .
Code Sample for Launch, by Adobe
GET /b/ss/[rsid]/0?vid=[UserID]&c.a.LaunchEvent=1&c.Intent=[intent]&pageName=[intent]  HTTP/1.1 
Host: sc.omtrdc.net 
Cache-Control: no-cache

Intents

Each of the digital assistants has algorithms that detect intents and then passes the intent down to the "App" so that the app knows what to do. These intents are a succinct representation of the request.
For example, if a user says, "Siri, Send John $20 for dinner last night from my banking app," the intent might be something like sendMoney .
By sending in each of these requests as an eVar, you will be able to run pathing reports on each of the intents for conversational apps. You will also want to make sure the "App" can handle requests without an intent. We recommend passing in No Intent Specified as opposed to leaving the value blank.
Code Sample
GET /b/ss/[rsid]/0?vid=[UserID]&c.a.AppID=Penmo1.0&c.a.LaunchEvent=1&c.Intent=SendPayment&pageName=[intent]  HTTP/1.1 
Host: sc.omtrdc.net 
Cache-Control: no-cache

or
GET /b/ss/[rsid]/0?vid=[UserID]&c.a.AppID=Penmo1.0&c.a.LaunchEvent=1&c.Intent=No_Intent_Specified&pageName=[intent]  HTTP/1.1 
Host: sc.omtrdc.net 
Cache-Control: no-cache

Parameters/Slots/Entities

In addition to the intent the digital assistant will often have a set of key value pairs that give the details of the intent. These are called slots, entities or parameters. For example:
"Siri, Send John $20 for dinner last night from my banking app" would have the following parameters:
  • Who = John
  • Amount = 20
  • Why = Dinner
There is usually a finite number of these with your app. To track these in Analytics, send them into context data and then map each of the parameters to an eVar.
Code Sample
GET /b/ss/[rsid]/0?vid=[UserID]&c.a.AppID=Penmo1.0=1&c.a.LaunchEvent=1&c.Intent=SendPayment&c.Amount=20.00&c.Reason=Dinner&c.ReceivingPerson=John&c.Intent=SendPayment&pageName=[intent]  HTTP/1.1 
Host: sc.omtrdc.net 
Cache-Control: no-cache

Session

Although not all apps will be revenue generating, it is important to think about what success looks like and include some measurements for it. Adobe Analytics can measure revenue, ad-impressions and other forms of success along with the user behavior.

Error States

Sometimes the Digital Assistant will provide the app with inputs that it doesn't know how to handle. For example.
"Siri, Send John 20 bags of coal for dinner last night from my banking app"
When this happens the app should ask for clarification. Additionally when it is responding to a request like this you should send Analytics an event that indicates the App has an error state along with an evar that specifies what type of error occurred.
Be sure to include errors where the inputs are not correct and errors where the "App" had a problem.
Code Sample
GET /b/ss/[rsid]/0/?vid=[UserID]&c.a.AppID=Penmo1.0&c.Error=1&c.ErrorName=InvalidCurrency&pageName=[intent] HTTP/1.1 
Host: sc.omtrdc.net 
Cache-Control: no-cache

Device Capabilities

While most of the platforms don't expose the actual device that the user spoke too. They do expose the capabilities of the device as the available interfaces (e.g. Audio, Screen, Video, etc). This is useful information because it defines the types of content that can be used when interacting with your users. When measuring the interfaces it is best to concatenate them (in alphabetical order).
Example: :Audio:Camera:Screen:Video:
Notice the trailing and leading colons. These help when creating segments (for example, give me all interactions with :Audio: capabilities).

Analytics Reporting for Digital Assistants

Once your Digital Assistant App is implemented you can use the full power of Adobe Analytics with it. Below are just a few examples of the things you can do with Analytics.

Monitoring Intents

Most Apps have several intents and different things you can do. You could easily use Analysis Workspace to keep track of the top intents by instances and by users
This lets you see which features are being used most often and can give you a view into the adoption of new features.

Requests with Errors

You will be able to monitor errors to see if there are common places where users are getting having problems.

Flow Between Events

One of the most powerful things to do is look at the flow of intents. This is helpful in two ways. First, you can look within a session for how people flow between intents in a conversation. Second, you can look at how people flow between intents over longer time frames to see how their usage of the "App" is evolving.

Example

Pre-installed app
Person Device Response Action / Intent Get Request Analytics Data
Play Spoofify
"okay playing Spoofify"
Play
GET /b/ss/[rsid]/0?vid=[UserID]&c.a.AppID=Spoofify1.0&c.a.LaunchEvent=1&c.Intent=Play&pageName=PlayApp  HTTP/1.1  Host: sc.omtrdc.net  Cache-Control: no-cache
  • Visitor ID
  • App Version
  • # of Launches
  • Intent
  • Response
Change song
"okay what song do you want?"
ChangeSong
GET /b/ss/[rsid]/0?vid=[UserID]&c.a.AppID=Spoofify1.0&c.Intent=ChangeSong&pageName= AskForSong  HTTP/1.1  Host: sc.omtrdc.net  Cache-Control: no-cache
  • Visitor ID
  • App Version
  • Intent
  • *blank playlist
  • Response
Play "My Heart Will Go On" by Celine Dion
"okay playing 'My Heart Will Go On' by Celine Dion"
ChangeSong
GET /b/ss/[rsid]/0?vid=[UserID]&c.a.AppID=Spoofify1.0&c.Intent=ChangePlaylist&pageName= ActionPlaySong&c.SongID=[012345]  HTTP/1.1  Host: sc.omtrdc.net  Cache-Control: no-cache
  • Visitor ID
  • App Version
  • Intent
  • Song ID
  • Response
Change playlist
"okay what playlist do you want?"
ChangePlaylist
GET /b/ss/[rsid]/0?vid=[UserID]&c.a.AppID=Spoofify1.0&c.Intent=ChangePlaylist&pageName= AskForPlaylist  HTTP/1.1  Host: sc.omtrdc.net  Cache-Control: no-cache
  • Visitor ID
  • App Version
  • Intent
  • *blank playlist
  • Response
Play Usher
"okay playing Usher"
ChangePlaylist
GET /b/ss/[rsid]/0?vid=[UserID]&c.a.AppID=Spoofify1.0&c.Intent=ChangePlaylist&pageName= ActionPlayPlaylist&c.Playlist=Usher  HTTP/1.1  Host: sc.omtrdc.net  Cache-Control: no-cache
  • Visitor ID
  • App Version
  • Intent
  • Playlist
  • Response
Turn music off
*no response, music turns off
Off
GET /b/ss/[rsid]/0?vid=[UserID]&c.a.AppID=Spoofify1.0&c.Intent=Off&pageName=TurnsOffMusic  HTTP/1.1  Host: sc.omtrdc.net  Cache-Control: no-cache
  • Visitor ID
  • App Version
  • Intent
  • Response
Requires user to install app
Person Device Response Action / Intent Get Request Analytics Data
Install Spoofify
*no response
Install
GET /b/ss/[rsid]/0?vid=[UserID]&amp;c.a.InstallEvent=1&c.a.InstallDate=2017-04-24&c.a.AppID=Spoofify1.0&c.OSType=Alexa&c.Intent=Install&pageName=Install  HTTP/1.1  Host: sc.omtrdc.net  Cache-Control: no-cache
  • Visitor ID
  • Date
  • Intent
  • OS Version
  • Response
Play Spoofify
"okay playing Spoofify"
Play
GET /b/ss/[rsid]/0?vid=[UserID]&c.a.AppID=Spoofify1.0&c.a.LaunchEvent=1&c.Intent=Play&pageName=PlayApp  HTTP/1.1  Host: sc.omtrdc.net  Cache-Control: no-cache
  • Visitor ID
  • App Version
  • # of Launches
  • Intent
  • Response
Change song
"okay what song do you want?"
ChangeSong
GET /b/ss/[rsid]/0?vid=[UserID]&c.a.AppID=Spoofify1.0&c.Intent=ChangeSong&pageName= AskForSong  HTTP/1.1  Host: sc.omtrdc.net  Cache-Control: no-cache
  • Visitor ID
  • App Version
  • Intent
  • *blank playlist
  • Response
Play "My Heart Will Go On" by Celine Dion
"okay playing 'My Heart Will Go On' by Celine Dion"
ChangeSong
GET /b/ss/[rsid]/0?vid=[UserID]&c.a.AppID=Spoofify1.0&c.Intent=ChangePlaylist&pageName= ActionPlaySong&c.SongID=[012345]  HTTP/1.1  Host: sc.omtrdc.net  Cache-Control: no-cache
  • Visitor ID
  • App Version
  • Intent
  • Song ID
  • Response
Change playlist
"okay what playlist do you want?"
ChangePlaylist
GET /b/ss/[rsid]/0?vid=[UserID]&c.a.AppID=Spoofify1.0&c.Intent=ChangePlaylist&pageName= AskForPlaylist  HTTP/1.1  Host: sc.omtrdc.net  Cache-Control: no-cache
  • Visitor ID
  • App Version
  • Intent
  • *blank playlist
  • Response
Play Usher
"okay playing Usher"
ChangePlaylist
GET /b/ss/[rsid]/0?vid=[UserID]&c.a.AppID=Spoofify1.0&c.Intent=ChangePlaylist&pageName= ActionPlayPlaylist&c.Playlist=Usher  HTTP/1.1  Host: sc.omtrdc.net  Cache-Control: no-cache
  • Visitor ID
  • App Version
  • Intent
  • Playlist
  • Response
Turn music off
*no response, music turns off
Off
GET /b/ss/[rsid]/0?vid=[UserID]&c.a.AppID=Spoofify1.0&c.Intent=Off&pageName=TurnsOffMusic  HTTP/1.1  Host: sc.omtrdc.net  Cache-Control: no-cache
  • Visitor ID
  • App Version
  • Intent
  • Response