Personalized gesture recognition for user interaction with assistant systems
AssigneeFacebook Technologies, LLC
InventorsXiaohu Liu, Paul Anthony Crook, Francislav P. Penov, Rajen Subba
Life of this patent
Estimating the term…
Abstract
In one embodiment, a method includes receiving a user request from a first user from a client system associated with a first user, wherein the user request comprise a gesture-input from the first user and a speech-input from the first user, determining an intent corresponding to the user request based on the gesture-input by a personalized gesture-classification model associated with the first user, executing one or more tasks based on the determined intent and the speech-input, and sending instructions for presenting execution results of the one or more tasks to the client system responsive the user request.
Claims20
A method comprising, by one or more computing systems:
- receiving, from a client system associated with a first user, a user request from the first user, wherein the user request comprise a gesture-input from the first user and a speech-input from the first user;
- determining, by a personalized gesture-classification model associated with the first user, an intent corresponding to the user request based on the gesture-input;
- sending, to the client system, instructions for prompting the first user for one or more user inputs;
- determining, by an updated personalized gesture-classification model associated with the first user, an updated intent, wherein the updated personalized gesture-classification model is generated based on the one or more user inputs;
- executing one or more tasks based on the updated intent and the speech-input;
- and
- sending, to the client system, instructions for presenting execution results of the one or more tasks responsive the user request.
One or more computer-readable non-transitory storage media embodying software that is operable when executed to:
- receive, from a client system associated with a first user, a user request from the first user, wherein the user request comprise a gesture-input from the first user and a speech-input from the first user;
- send, to the client system, instructions for prompting the first user for one or more user inputs;
- determine, by an updated personalized gesture-classification model associated with the first user, an updated intent, wherein the updated personalized gesture-classification model is generated based on the one or more user inputs;
- execute one or more tasks based on the updated intent and the speech-input;
- and
- send, to the client system, instructions for presenting execution results of the one or more tasks responsive the user request.
A system comprising:
- one or more processors;
- and a non-transitory memory coupled to the processors comprising instructions executable by the processors, the processors operable when executing the instructions to:
- receive, from a client system associated with a first user, a user request from the first user, wherein the user request comprise a gesture-input from the first user and a speech-input from the first user;
- send, to the client system, instructions for prompting the first user for one or more user inputs;
- determine, by an updated personalized gesture-classification model associated with the first user, an updated intent, wherein the updated personalized gesture-classification model is generated based on the one or more user inputs;
- execute one or more tasks based on the updated intent and the speech-input;
- and
- send, to the client system, instructions for presenting execution results of the one or more tasks responsive the user request.
Dependencies are inferred from the claim text.