US 9,959,129 B2Granted

Headless task completion within digital personal assistants

AssigneeMicrosoft Technology Licensing, LLC

InventorsVishwac Sena Kannan, Aleksandar Uzelac, Daniel J. Hwang, Robert L. Chambers, Thomas Soemo, Adina Magdalena Trufinescu, Khuram Shahid, Ali Emami

Life of this patent

Estimating the term…

patentgrep

Loading patent…

Abstract

Techniques are described for headlessly completing a task of an application in the background of a digital personal assistant. For example, a method can include receiving a voice input via a microphone. Natural language processing can be performed using the voice input to determine a user voice command. The user voice command can include a request to perform a task of the application. The application can be caused to execute the task as a background process without a user interface of the application appearing. A user interface of the digital personal assistant can provide a response to the user, based on a received state associated with the task, so that the response comes from within a context of the user interface of the digital personal assistant without surfacing the user interface of the application.

Claims19

Dependent claims

A computing device comprising:

a processing unit;
memory;
one or more microphones;
and
a speech-controlled digital personal assistant configured to perform operations comprising:
receiving speech input generated by a user via the one or more microphones;
performing speech recognition using the speech input to determine a spoken command, wherein the spoken command comprises a request to perform a task of a third-party application, and wherein the task is identified using a data structure that defines tasks of third-party applications invokable by spoken command, the third-party applications comprising tasks that are capable of being executed using user interfaces of the third-party applications and tasks that are capable of being headlessly executed without using the user interfaces of the third-party applications, and the data structure defining how the task is capable of being executed by the digital personal assistant;
using the data structure to select between headlessly executing the task of the third-party application and executing the task using the user interface of the third-party application;
causing the third-party application to execute as a background process to headlessly execute the task when headless execution of the task of the third-party application is selected;
receiving a response from the third-party application indicating a state associated with the task;
and
providing, by a user interface of the speech-controlled digital personal assistant, a response to the user based on the received state associated with the task so that the response comes from within a context of the user interface of the speech-controlled digital personal assistant without surfacing the user interface of the third-party application.

9 more elements

A method, implemented by a computing device comprising a microphone, the method comprising:

receiving, by a voice-controlled digital personal assistant, a digital voice input generated by a user, wherein the digital voice input is received via the microphone;
performing natural language processing using the digital voice input to determine a user voice command, wherein the user voice command comprises a request to perform a pre-defined function of a third-party voice-enabled application, and wherein the pre-defined function is identified using a data structure that defines functions supported by available third-party voice-enabled applications using voice input, the third-party voice-enabled applications comprising pre-defined functions that are capable of being executed using user interfaces of the third-party voice-enabled applications and pre-defined functions that are capable of being headlessly executed without using the user interfaces of the third-party voice-enabled applications, and the data structure defining how the pre-defined function is capable of being executed by the digital personal assistant;
in response to determining the user voice command comprising the request to perform the pre-defined function of the third-party application, using the data structure to select between headlessly executing the pre-defined function of the third-party voice-enabled application and executing the pre-defined function using the user interface of the third-party voice-enabled application;
causing the third-party voice-enabled application to execute the pre-defined function as a background process without a user interface of the third-party voice-enabled application appearing on a display of the computing device;
receiving a response from the third-party voice-enabled application indicating a state associated with the pre-defined function;
and
providing, by a user interface of the voice-controlled digital personal assistant, a response to the user based on the received state associated with the pre-defined function so that the response comes from within a context of the user interface of the voice-controlled digital personal assistant without surfacing the user interface of the third-party voice-enabled application.

4 more elements

A computer-readable storage device storing computer-executable instructions for causing a computing device to perform operations for completing a task of a voice-enabled application within the context of a voice-controlled digital personal assistant, the operations comprising:

receiving, by the voice-controlled digital personal assistant, a digital voice input generated by a user, wherein the digital voice input is received via a microphone;
performing natural language processing using the digital voice input to determine a user voice command, wherein the user voice command comprises a request to perform the task of the voice-enabled application, and wherein the task is identified using an extensible data structure that maps user voice commands to tasks of voice-enabled applications, the voice-enabled applications comprising tasks that are capable of being executed using user interfaces of the voice-enabled applications and tasks that are capable of being headlessly executed without using the user interfaces of the voice-enabled applications, and the data structure defining how the task is capable of being executed by the digital personal assistant;
in response to determining the user voice command, using the data structure to select between executing the task of the voice-enabled application as a foreground task or as a background task;
when the task is selected to be executed as a background task, causing the voice-enabled application to execute the task as a background task and within a context of the voice-controlled digital personal assistant without a user interface of the voice-enabled application surfacing;
receiving a response from the voice-enabled application indicating a state associated with the task;
and
providing a response to the user based on the received state associated with the task, the response provided within the context of the voice-controlled digital personal assistant without a user interface of the voice-enabled application surfacing when it is determined that the task is a background task.

4 more elements

Dependencies are inferred from the claim text.

Facts

Application: 14/593,584Filed 9 Jan 2015
Published: US 2016/0203002 A114 Jul 2016
Granted: 1 May 2018
Term ends: Not estimatedThe source records don’t support an estimate.

Classification10 codes

G06F 9/453
Execution arrangements for user interfaces: help systems
G06F 3/0484
Interaction techniques based on graphical user interfaces [GUI]: for the control of specific functions or operations, e.g. selecting or…
G06F 3/162
Sound input; Sound output: interface to dedicated audio devices, e.g. audio drivers, interface to CODECs
G06F 3/167
Sound input; Sound output: audio in a user interface, e.g. using voice commands for navigating, audio feedback