US 10,242,393 B1Granted

Determine an item and user action in a materials handling facility

AssigneeAmazon Technologies, Inc.

InventorsDilip Kumar, Gianna Lise Puerini, Jason Michael Famularo, Amber Autrey Taylor, Thomas Meilandt Mathiesen, Jared Joseph Frank

Life of this patent

Estimating the term…

patentgrep

Loading patent…

Abstract

Described is a system and method for presenting event information to a user and, if necessary, obtaining confirmation of different aspects (user, item, action) of the event. In some implementations, an event includes a user, an action, and an item. For example, an event may include a user picking an item from an inventory location, a user placing an item into a tote associated with the user, etc. if the aspects of the event cannot be determined with a high enough degree of confidence, a user interface may be generated and sent to the user requesting confirmation of one or more of the aspects of the event.

Claims13

Dependent claims

A system, comprising:

a plurality of cameras positioned within a materials handling facility, each camera of the plurality of cameras configured to generate image data of objects located within the materials handling facility;
one or more processors;
and
a memory coupled to the one or more processors and storing program instructions that when executed by the one or more processors cause the one or more processors to at least:
detect an occurrence of an event performed by a user within a materials handling facility;
receive, from at least one of the plurality of cameras, an image of the user performing the event, wherein the image includes at least a portion of a representation of the user and at least a portion of a representation of an item;
process the image to determine a user identity of the user represented in the image;
determine a user identification confidence score representative of a confidence that the user identity of the user is correct;
determine an action determination confidence score representative of a confidence that a determined action performed by the user as part of the event is correct;
process the image to determine an item identity of the item represented in the image;
determine an item determination confidence score representative of a confidence that the item identity of the item is correct;
determine that the item determination confidence score does not exceed an item confidence threshold;
and
send data for use in presenting to the user a user interface identifying the user identity, the action and the item identity, wherein:
the presentation on the user interface includes a request that the user confirm the item identity of the item involved in the event is correct;
and
the action is at least one of an item pick from an inventory location, an item place at the inventory location, an item pick from a tote, or an item place into the tote.

14 more elements

A system, comprising:

a plurality of cameras positioned within a materials handling facility, each camera of the plurality of cameras configured to generate image data of objects located within the materials handling facility;
one or more processors;
and
a memory coupled to the one or more processors and storing program instructions that when executed by the one or more processors cause the one or more processors to at least:
determine that a user has performed an action;
receive, from at least one camera of the plurality of cameras, image data of the user performing the action;
process the image data to determine a user identity of the user and an item identity of an item;
determine:
a user identification confidence score representative of a confidence that the user identity of the user is correct;
an action determination confidence score representative of a confidence that a determined action performed by the user is correct;
and
an item determination confidence score representative of a confidence that the item identity of the item involved in the action is correct;
determine that at least one of the action determination confidence score or the item determination confidence score does not exceed a threshold;
and
send for presentation to the user:
item information that includes a representation of the item;
action information representative of the action performed by the user;
and
a request that the user confirm at least one of:
the action performed by the user;
the item identity;
or that the user did not perform the action, wherein the request is based at least in part on one or more of the determination that at least one of the action determination confidence score or the item determination confidence score do not exceed a threshold.

19 more elements

Dependencies are inferred from the claim text.