Today, during Alexa Live, a virtual event for Alexa vendors and developer partners, Amazon introduced tools and resources designed to enable new Alexa voice app experiences. Among others, the company launched deep neural networks aimed at making Alexa’s natural language understanding more accurate for custom apps, as well as an API that enables the use of web technologies to create gaming apps for select Alexa devices. Amazon also launched Alexa Conversations in beta, a deep learning-based way to help developers create more natural applications with fewer lines of code. And it introduced a new preview service, Alexa for Apps, which allows Alexa apps to trigger actions like searches in smartphone apps.
The disclosures occur when the pandemic supercharges the use of the voice application, which was already on the rise. According to a study by NPR and Edison Research, the percentage of voice-enabled device owners who use commands at least once a day increased between early 2020 and early April. Just over a third of smart speaker owners say they listen to more music, entertainment, and news from their devices than before, and owners report that this year they requested an average of 10.8 tasks per week from their assistant this year compared to 9.4 different tasks in 2019.
Amazon says that deep neural networks for natural language understanding improve the precision of intention and recognition of the value of space by 15% on average. Attempts represent actions that satisfy user requests, and specify names and expressions that a user would say to invoke the attempt. Slot values are intentional arguments such as dates, phrases, and item lists. “This essentially changes the modeling technology used by Alexa apps behind the scenes,” Nedim Fresko, vice president of devices for Alexa, told VentureBeat in a phone interview. “We are expanding it to cover more applications … that are available.”
The use of deep neural networks, which can currently be generalized from phrases like “buy me an apple” to “order an orange for me,” will expand to 400 eligible skills in the US, Britain, India, and Germany to later this year, according to Amazon.
Thanks to the new NFI Toolkit (in preview), developers can choose to provide Alexa with additional cues on requests that their applications can handle. For example, they can provide alternate start phrases that customers can use to launch the app and intentions that Alexa can keep in mind when routing unnamed requests, and then see the routes that customers use to invoke the app from a dashboard. Fresko says that early users have seen a 15% increase in usage.
Alexa Conversations
Alexa Conversations, which was announced last June in the developer preview at Amazon’s re: MARS conference, reduces the lines of code needed to create voice apps from 5,500 to approximately 1,700. Leveraging artificial intelligence to better understand intentions and declarations so developers don’t have to define them, Amazon also says Conversations reduces Alexa interactions that could have led 40 exchanges to a dozen or so.
The conversation dialog manager is powered by two innovations, according to Amazon: a dialogue simulator and a “conversation first” modeling architecture. Dialog simulator generalizes a small number of sample dialogs provided by a developer into tens of thousands of annotated dialogs, while the modeling architecture leverages generated dialogs to train deep learning based models to support dialogues beyond simple trails provided by sample dialogues.
Developers provide things like API access and entities that the API has access to, in effect, describe the functionality of the application. After these and some sample swaps are given, the Conversations dialog box manager can extrapolate the possible twists of the dialog box.
The first conversation use case, unveiled last year, seamlessly brought Alexa apps together to allow people to buy movie tickets, set up rides, and book dinner reservations. (OpenTable, Uber, and Atom Tickets were among the earliest users of Conversations.) In light of the pandemic, that scenario seems less useful. But Fresko said it simply illustrates how conversations can combine elements of multiple applications without much effort on the part of developers; Companies like iRobot and Philosophical Creations (published by the Big Sky app) are already using it.
“The dialogues are really difficult to emulate with brute force techniques. Developers typically resort to dialog trees and flowcharts to anticipate every turn the conversation can take, and the complexity can be disproportionate, “said Fresko.” With Conversations, you don’t have a manual compilation context, just what we will do for you. “
‘Immersive’ audio and visuals
Alexa Presentation Language (APL), a set of tools designed to make it easier for developers to create visual applications for Alexa, is expanding to sound with APL for Audio. APL for Audio includes new mixing capabilities that support audio and soundscape creation in Alexa apps; Audio can be mixed with Alexa speech, multiple voices can be mixed together with sound effects, or images can be synchronized with clips that dynamically respond to users.
“This reflects the reality that Alexa has become useful not only in speakers but also in a variety of devices,” said Fresko. “It’s a huge improvement in workflow for developers, particularly environment or meditation app developers, that sort of thing.”
Joining APL for Audio is the web API for games, which makes open standards like Canvas 2D, WebAudio, WebGL, JavaScript and CSS available to Alexa developers. On Echo Show and on select Fire TV devices, developers can use the web API for games to create experiences that launch web applications, which are displayed on the device to handle voice requests and react to local events such as listening and muting the microphone . End users can interact with the web application through voice, touch, or remote controls (on Fire TV).
Very active
Launched this week in preview, the new Skills Resume feature allows developers to experiment with apps running in the background on Alexa devices. It maintains the logic of an application intact to allow customers to engage with it as needed for an extended period of time or continue with it where they left off.
Fresko gave this example: A user tells the Uber app to have Alexa call a car, then switches from the Uber app to music, weather report, and news. As the car approaches, the Uber app returns to the surface to notify you. “Skills resume … allows apps to proactively inform users from the background,” said Fresko. “Think of meditation or training applications that keep a timer while the user performs other tasks.”
Skill Resume fits in with Alexa for Apps, which integrates the content and functionality of iOS and Android apps with Alexa. Through deep links, developers can assign tasks like opening the home page of a mobile app, presenting search results, and other key features to voice commands from the Alexa app. A yellow pages-type app could take advantage of deep links to extract restaurant information when a user asks Alexa about it, Fresko explained, while a camera app could link an Alexa command to the shutter button. TikTok editor ByteDance worked with Amazon to support the “Alexa, ask TikTok to start my recording” command.
By using Quick Links for Alexa (in beta for US English and US Spanish), developers can further take advantage of deep links to drive traffic to voice apps from websites and mobile apps . They can establish deep links to specific content in their applications using URL query string parameters and add attribution parameters to measure the performance of the online advertising campaign. “This makes it easier for customers to find skills and for developers to promote their skills in a variety of media. We hope it will lead to new opportunities, ”said Fresko.
It was also announced today: In select regions, customers can now purchase premium content in-app, such as expansion packs, monthly subscriptions, and supplies, on Amazon.com and on Echo device displays with displays. Previously, the only way to make those purchases was through voice. (Amazon remains tight-lipped about how much consumers spend on Alexa skills, but by some estimates, it’s at least $ 2 billion per year.)
