Method and system for understanding and fulfilling verb-based succinct queries
AssigneeGoogle Inc.
InventorsKeun Soo Yim, Adam Coimbra
Life of this patent
Estimating the term…
Abstract
Some implementations receive audio data capturing a verb-based succinct query that includes an action but is void of any app entity and determine, based on processing of the audio data, the action and one or more instance candidates associated with the action. Some of those implementations further query an app entity database for usage information and/or capability information of one or more app entities that correspond to the one or more instance candidates and generate, based on selecting from the one or more app entities, a response or action responsive to the verb-based succinct query.
Claims19
A method implemented by one or more processors, the method comprising:
- receiving audio data capturing a user query that includes an action term but is void of any application entity that is descriptive of an application to which the action term is directed, the audio data being detected via one or more microphones of a client device;
- processing the audio data to generate a transcription of the user query;
- processing, using a natural language understanding engine, the transcription of the user query to determine an action that corresponds to the action term and a plurality of entity instance candidates for the action, each of the entity instance candidates being for a corresponding one of a plurality of applications accessible via the client device;
- querying an application entity database storing a set of application entities, for usage information of application entities that correspond to the plurality of entity instance candidates, wherein the application entities are stored in the application entity database based on being donated by multiple of the plurality of applications accessible via the client device;
- generating a model input based on the user query and the usage information of the application entities;
- processing, using a trained machine learning model, the model input to generate a model output;
- and
- causing, based on the model output, one or more actions to be performed by at least one of the plurality of applications accessible via the client device.
A method implemented by one or more processors, the method comprising:
- receiving audio data capturing a user query that includes an action term but is void of any application entity that is descriptive of an application to which the action term is directed, the audio data being detected via one or more microphones of a client device;
- processing the audio data to generate a transcription of the user query;
- processing, using a natural language understanding engine, the transcription of the user query to determine an action that corresponds to the action term and a plurality of entity instance candidates for the action, each of the entity instance candidates being for a corresponding one of a plurality of applications accessible via the client device;
- querying an application entity database storing a set of application entities, for usage information of application entities that correspond to the plurality of entity instance candidates, wherein the application entities are stored in the application entity database based on being donated by multiple of the plurality of applications accessible via the client device;
- generating, based on the user query and the usage information of the application entities that correspond to the plurality of entity instance candidates, one or more actions to be performed;
- and
- causing the one or more actions to be performed by at least one of the plurality of applications accessible via the client device.
Dependencies are inferred from the claim text.