US 10,885,129 B2Granted

Using frames for action dialogs

AssigneeGoogle Inc.

InventorsDavid P. Whipp, David Kliger Elson, Shir Judith Yehoshua

Life of this patent

Estimating the term…

Loading patent…

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using frames for performing tasks. One of the methods includes receiving a first request to perform a task, the first request comprising user speech identifying the task; generating a frame associated with the task, wherein the frame comprises one or more types of values necessary to perform the task, and wherein each type of value can be satisfied by a respective value; receiving a second request to provide information related to a question, the second request comprising user speech identifying the question; providing information identifying the question to a search engine, and receiving a response identifying one or more terms; determining that at least one term can satisfy a type of value necessary to perform the task; and storing the at least one term in the frame.

Claims19

Dependent claims

A method performed by one or more computers of an automated spoken dialog system, the method comprising:

receiving, by the one or more computers and from a user device, a first request to perform a task, wherein the first request comprises user speech identifying the task;
in response to receiving the first request, generating, by the one or more computers, a first frame associated with the task, wherein the first frame specifies one or more types of values used to perform the task;
providing, by the one or more computers and to the user device, a prompt associated with the first request, wherein the prompt requests user input of a particular type of value from among the one or more types of values specified by the first frame;
in a response to the prompt and before receiving user input of the particular type of value requested by the prompt, receiving, by the one or more computers and from the user device, a second request including a question that requests information from the one or more computers, the second request comprising user speech identifying the question;
in response to receiving the second request:
generating, by the one or more computers, a second frame representing the question that requests information, wherein the second frame is different from the first frame and specifies one or more types of values needed to respond to the question, wherein at least one of the one or more types of values corresponds to the particular type of value requested by the prompt,
determining, by the one or more computers, that the question requests information inclusive of user data from the user device, and
providing, by the one or more computers, information identifying the question to a search engine;
receiving, by the one or more computers, a response from the search engine that identifies one or more results, wherein the one or more results provide information inclusive of the user data from the user device that is responsive to the question;
determining, by the one or more computers, that a particular result from among the one or more results provides a value of the particular type of value specified by the second frame;
subsequent to determining that the particular result provides the value of the particular type of value specified by the second frame:
determining, by the one or more computers, that the value corresponds to the particular type of value requested by the prompt;
in response to determining that the particular result provides a value of the particular type of value requested by the prompt;
storing, by the one or more computers and in the first frame associated with the task requested by the first request, the value of the particular type of value that was provided by the particular result, and
deleting, by the one or more computers, the second frame, and
using, by the one or more computers, the stored value in the first frame to carry out the task requested by the first request.

13 more elements

A system comprising:

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
receiving, by the one or more computers and from a user device, a first request to perform a task, wherein the first request comprises user speech identifying the task;
in response to receiving the first request, generating, by the one or more computers, a first frame associated with the task, wherein the first frame specifies one or more types of values used to perform the task;
providing, by the one or more computers and to the user device, a prompt associated with the first request, wherein the prompt requests user input of a particular type of value from among the one or more types of values specified by the first frame;
in a response to the prompt and before receiving user input of the particular type of value requested by the prompt, receiving, by the one or more computers and from the user device, a second request including a question that requests information from the one or more computers, the second request comprising user speech identifying the question;
in response to receiving the second request:
generating, by the one or more computers, a second frame representing the question that requests information, wherein the second frame is different from the first frame and specifies one or more types of values needed to respond to the question, wherein at least one of the one or more types of values corresponds to the particular type of value requested by the prompt,
determining, by the one or more computers, that the question requests information inclusive of user data from the user device, and
providing, by the one or more computers, information identifying the question to a search engine;
receiving, by the one or more computers, a response from the search engine that identifies one or more results, wherein the one or more results provide information inclusive of the user data from the user device that is responsive to the question;
determining, by the one or more computers, that a particular result from among the one or more results provides a value of the particular type of value specified by the second frame;
subsequent to determining that the particular result provides the value of the particular type of value specified by the second frame:
determining, by the one or more computers, that the value corresponds to the particular type of value requested by the prompt;
in response to determining that the particular result provides a value of the particular type of value requested by the prompt:
storing, by the one or more computers and in the first frame associated with the task requested by the first request, the value of the particular type of value that was provided by the particular result, and
deleting, by the one or more computers, the second frame;
and
using, by the one or more computers, the stored value in the first frame to carry out the task requested by the first request.

15 more elements

One or more non-transitory computer-readable storage media encoded with instructions that, when executed by one or more computers, cause the one or more computers to perform operations comprising:

receiving, by the one or more computers and from a user device, a first request to perform a task, wherein the first request comprises user speech identifying the task;
in response to receiving the first request, generating, by the one or more computers, a first frame associated with the task, wherein the first frame specifies one or more types of values used to perform the task;
providing, by the one or more computers and to the user device, a prompt associated with the first request, wherein the prompt requests user input of a particular type of value from among the one or more types of values specified by the first frame;
in a response to the prompt and before receiving user input of the particular type of value requested by the prompt, receiving, by the one or more computers and from the user device, a second request including a question that requests information from the one or more computers, the second request comprising user speech identifying the question;
in response to receiving the second request:
generating, by the one or more computers, a second frame representing the question that requests information, wherein the second frame is different from the first frame and specifies one or more types of values needed to respond to the question, wherein at least one of the one or more types of values corresponds to the particular type of value requested by the prompt,
determining, by the one or more computers, that the question requests information inclusive of user data from the user device, and
providing, by the one or more computers, information identifying the question to a search engine;
receiving, by the one or more computers, a response from the search engine that identifies one or more results, wherein the one or more results provide information inclusive of the user data from the user device that is responsive to the question;
determining, by the one or more computers, that a particular result from among the one or more results provides a value of the particular type of value specified by the second frame;
subsequent to determining that the particular result provides the value of the particular type of value specified by the second frame:
determining, by the one or more computers, that the value corresponds to the particular type of value requested by the prompt;
in response to determining that the particular result provides a value of the particular type of value requested by the prompt;
storing, by the one or more computers and in the first frame associated with the task requested by the first request, the value of the particular type of value that was provided by the particular result, and
deleting, by the one or more computers, the second frame;
and
using, by the one or more computers, the stored value in the first frame to carry out the task requested by the first request.

14 more elements

Dependencies are inferred from the claim text.

Facts

Application: 14/881,778Filed 13 Oct 2015 · us provisional application
Priority: 10 Dec 2014
Published: US 2016/0171114 A116 Jun 2016
Granted: 5 Jan 2021
Term ends: Not estimatedThe source records don’t support an estimate.

Classification14 codes

G06F 16/9538
Querying, e.g. by the use of web search engines: presentation of query results
G06F 3/167
Sound input; Sound output: audio in a user interface, e.g. using voice commands for navigating, audio feedback
G06F 16/243
Query formulation: natural language query formulation
G06F 16/3329
Query formulation: natural language query formulation