US 10,503,468 B2Granted

Voice enabling applications

AssigneeAmazon Technologies, Inc.

InventorsManikandan Thangarathnam, Suraj Gopalakrishnan

Life of this patent

Estimating the term…

Loading patent…

Abstract

Systems and methods for voice control of computing devices are disclosed. Applications may be downloaded and/or accessed by a device having a display, and content associated with the applications may be displayed. Many applications do not allow for voice commands to be utilized to interact with the displayed content. Improvements described herein allow for non-voice-enabled applications to utilize voice commands to interact with displayed content by determining screen data displayed by the device and utilizing the screen data to determine an intent associated with the application. Directive data to perform an action corresponding to the intent may be sent to the device and may be utilized to perform the action on an object associated with the displayed content.

Claims20

Dependent claims

A system comprising:

one or more processors;
and
non-transitory computer-readable media storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
receiving audio data representing a user utterance;
determining intent data based at least in part on the audio data;
identifying first directive data corresponding to the intent data, the first directive data configured to, when sent to a device, cause the device to perform a first operation with respect to a portion of content displayed by the device;
identifying second directive data corresponding to the intent data, the second directive data configured to, when sent to the device, cause the device to perform a second operation with respect to the portion of the content;
determining a first priority associated with the first directive data from a first content type associated with the portion of the content, the first content type being text content;
determining a second priority associated with the second directive data from a second content type associated with the portion of the content, the second content type being icon content;
determining that the first directive data is prioritized over the second directive data based on the first content type being prioritized over the second content type;
selecting the first directive data based on the first directive data being prioritized over the second directive data;
and
sending the first directive data to the device to cause an action to be performed with respect to the portion of the content.

10 more elements

A method, comprising:

receiving audio data representing a user utterance;
determining, based at least in part on the audio data, intent data associated with the user utterance;
identifying first directive data corresponding to the intent data, the first directive data configured to, when received by a device, cause the device to perform a first operation with respect to a portion of content of an application;
identifying second directive data corresponding to the intent data, the second directive data configured to, when received by the device, cause the device to perform a second operation with respect to the portion of the content;
determining that the first directive data is prioritized over the second directive data;
selecting the first directive data based at least in part on the first directive data being prioritized over the second directive data;
and
sending the first directive data to the device to cause an action to be performed on the portion of the content.

5 more elements

A system, comprising:

one or more processors;
and
non-transitory computer-readable media storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
determining, based at least in part on audio data representing a user utterance, intent data associated with the user utterance;
identifying first directive data corresponding to the intent data, the first directive data configured to, when sent to a device, cause the device to perform a first operation with respect to a portion of content of an application;
identifying second directive data corresponding to the intent data, the second directive data configured to, when sent to the device, cause the device to perform a second operation with respect to the portion of the content;
determining that the first directive data is prioritized over the second directive data;
and
sending the first directive data to the device based at least in part on determining that the first directive data is prioritized over the second directive data.

6 more elements

Dependencies are inferred from the claim text.

Facts

Application: 15/836,428Filed 8 Dec 2017
Published: US 2019/0179606 A113 Jun 2019
Granted: 10 Dec 2019
Term ends: Not estimatedThe source records don’t support an estimate.

Classification7 codes

G06F 3/167
Sound input; Sound output: audio in a user interface, e.g. using voice commands for navigating, audio feedback
G06F 40/295
Phrasal analysis, e.g. finite state techniques or chunking: named entity recognition
G06F 40/30
Handling natural language data: semantic analysis
G10L 15/22
Speech recognition: procedures used during a speech recognition process, e.g. man-machine dialogue

Family

US 2019/0179606 A1Pre-grant publication · published 13 Jun 2019
US 10,503,468 B2This patent

From the US filing history. Foreign family members aren’t listed.

Timeline

Citations

Open elsewhere

Building an agent or pulling many patents? Use the API or MCP server instead of scraping pages. Get an API key