Back to results
US 11,232,509 B1Granted
Expression and gesture based assistance
AssigneeAmazon Technologies, Inc.
InventorsMuralidhar Koka, Sudarshan Narasimha Raghavan, Yasser Baseer Asmi, Ammar Chinoy, Korwin Jon Smith, Dilip Kumar
Life of this patent
Estimating the term…
Abstract
This disclosure describes a system for assisting users in locating items from inventory locations and/or obtaining additional information about items. In some instances, a user may experience difficulty in locating an inventory item in a materials handling facility. By detecting one or more expressions presented by the user, an inventory management system may determine that assistance is to be provided to the user, determine and provide the appropriate assistance to the user.
Claims26
A method, comprising:
- under control of one or more computing devices configured with executable instructions,
- identifying a user within a materials handling facility;
- detecting a facial expression of emotion presented by the user, wherein the facial expression of emotion is at least one of a macroexpression or a microexpression;
- in response to detecting the facial expression of emotion, increasing an expression count for the user;
- determining, based at least in part on one or more of a position of the user within the materials handling facility, the user, or a user characteristic, a threshold, wherein the threshold varies for different users based on one or more of the position of the user within the materials handling facility, the user, or the user characteristic;
- determining that the expression count exceeds the threshold;
- in response to determining that the expression count exceeds the threshold, determining that assistance is to be provided to the user;
- determining an assistance to be provided to the user based at least in part on the facial expression of emotion and one or more of a prior interaction with the user, an item for the user, a user history, a movement of the user, a location of the user within the materials handling facility, a user preference, a user profile, an expression count corresponding to the expression, or at least one item selected by the user;
- and
- causing assistance to be provided to the user.
7 more elements
A system, comprising:
- a plurality of cameras positioned within a materials handling facility and remote from a user, each camera of the plurality of cameras configured to generate images of objects located within the materials handling facility;
- one or more processors;
- and
- a memory coupled to the one or more processors and storing program instructions that when executed by the one or more processors cause the one or more processors to at least:
- monitor a position of a user within a materials handling facility;
- detect a facial expression of emotion presented by the user, wherein the facial expression of emotion is at least one of a macroexpression or a microexpression;
- in response to detection of the facial expression of emotion, increase an expression count for the user;
- determine, based at least in part on one or more of the position of the user within the materials handling facility, the user, or a user characteristic, a threshold, wherein the threshold varies for different users based on one or more of the position of the user within the materials handling facility, the user, or the user characteristic;
- determine that the expression count exceeds the threshold;
- in response to determination that the expression count exceeds the threshold and based at least in part on the position of the user, determine that assistance is to be provided to the user;
- determine an assistance to be provided to the user based at least in part on one or more of the facial expression of emotion, a prior interaction with the user, an item for the user, a user history, a movement of the user, or at least one item selected by the user;
- and
- cause the assistance to be provided to the user.
10 more elements
A computer-implemented method for providing assistance to users within a materials handling facility, comprising:
- under control of one or more computing systems configured with executable instructions,
- monitoring a movement of a user through a materials handling facility;
- detecting a facial expression of emotion presented by the user, wherein the facial expression of emotion is at least one of a macroexpression or a microexpression;
- in response to detecting the facial expression of emotion, increasing an expression count for the user;
- determining, based at least in part on one or more of a position of the user within the materials handling facility, the user, or a user characteristic a threshold, wherein the threshold varies for different users based on one or more of the position of the user within the materials handling facility, the user, or the user characteristic;
- determining that the expression count exceeds the threshold;
- in response to determining that the expression count exceeds the threshold and based at least in part on the movement, determining that assistance is to be provided to the user;
- determining an assistance to be provided to the user based at least in part on one or more of the facial expression of emotion, a prior interaction with the user, an item for the user, a user history, the movement of the user, or at least one item selected by the user;
- and
- causing the assistance to be provided to the user.
7 more elements
Dependencies are inferred from the claim text.