US 11,308,442 B1Granted

Joint inference determination of interaction using data from weight sensors

AssigneeAmazon Technologies, Inc.

InventorsDilip Kumar, Liefeng Bo, Nikhil Chacko, Robert Crandall, Nishitkumar Ashokkumar Desai, Jayakrishnan Kumar Eledath, Gopi Prashanth Gopal, Gerard Guy Medioni, Paul Eugene Munger, Kushagra Srivastava

Life of this patent

Estimating the term…

patentgrep

Loading patent…

Abstract

Sensor data from load cells at a shelf is processed using a first time window to produce first event data describing coarse and sub-events. Location data is determined that indicates where on the shelf weight changes occurred at particular times. Hypotheses are generated using information about where items are stowed, weights of those of items, type of event, and the location data. If confidence values of these hypotheses are below a threshold value, second event data is determined by merging adjacent sub-events. This second event data is then used to determine second hypotheses which are then assessed. A hypothesis with a high confidence value is used to generate interaction data indicative of picks or places of particular quantities of particular types of items from the shelf.

Claims20

Dependent claims

A system comprising:

a shelf to stow one or more types of items in one or more areas;
a plurality of weight sensors arranged at different positions with respect to the shelf, wherein the plurality of weight sensors generate sensor data indicative of a weight of the shelf at the different positions;
and
one or more processors to execute instructions to:
access the sensor data obtained from the plurality of weight sensors;
determine, based on at least a portion of the sensor data indicative of the weight of the shelf at the different positions, a set of event data comprising:
first event data that is indicative of a first event comprising a first weight change that occurs from a first time to a second time;
second event data that is indicative of a second event comprising a second weight change that occurs from a third time to a fourth time, wherein the third time and the fourth time occur between the first time and the second time;
and
third event data that is indicative of a third event comprising a third weight change that occurs from a fifth time to a sixth time, wherein the fifth time and the sixth time occur between the first time and the second time, and the fifth time is adjacent in time to the fourth time;
determine a first set of event hypotheses based on the second event data, wherein each event hypothesis of the first set of event hypotheses has a first confidence value;
determine a second set of event hypotheses based on the third event data, wherein each event hypothesis of the second set of event hypotheses has a second confidence value;
determine that the first confidence value of a highest ranking hypothesis in the first set of event hypotheses is less than a first threshold value;
determine that the second confidence value of a highest ranking hypothesis in the second set of event hypotheses is less than the first threshold value;
determine, based on the at least a portion of the sensor data, fourth event data that is indicative of a fourth weight change that occurs from the third time to the sixth time;
determine a third set of event hypotheses based on the fourth event data, wherein:
a first event hypothesis of the third set of event hypotheses is indicative of movement of at least one first item and is associated with a third confidence value;
and
a second event hypothesis of the third set of event hypotheses is indicative of movement of at least one second item and is associated with a fourth confidence value;
and
generate output data indicative of the at least one second item based on the third confidence value and the fourth confidence value.

18 more elements

A system comprising:

a fixture having:
a first area and a second area;
and
a plurality of weight sensors arranged at different positions with respect to the first area and the second area, wherein the plurality of weight sensors generate sensor data indicative of one or more weights of items stowed at the first area and the second area;
and
one or more processors to execute instructions to:
access the sensor data obtained from the plurality of weight sensors;
determine, based on at least a portion of the sensor data indicative of the one or more weights of the items stowed at the first area and the second area, a first set of event data comprising:
first event data that is indicative of a first event comprising a first weight change that occurs from a first time to a second time;
second event data that is indicative of a second event comprising a second weight change that occurs from a third time to a fourth time, wherein the third time and the fourth time occur between the first time and the second time;
and
third event data that is indicative of a third event comprising a third weight change that occurs from a fifth time to a sixth time, wherein the fifth time and the sixth time occur between the first time and the second time, and the fifth time is adjacent in time to the fourth time;
determine a first set of event hypotheses based on the second event data, wherein each event hypothesis of the first set of event hypotheses has a first confidence value;
determine a second set of event hypotheses based on the third event data, wherein each event hypothesis of the second set of event hypotheses has a second confidence value;
determine, based on the at least a portion of the sensor data and one or more of the first confidence value or the second confidence value, fourth event data that is indicative of a fourth weight change that occurs from the third time to the sixth time;
determine a third set of event hypotheses based on the fourth event data, wherein:
a first event hypothesis of the third set of event hypotheses is indicative of movement of at least one first item and is associated with a third confidence value;
and
a second event hypothesis of the third set of event hypotheses is indicative of movement of at least one second item and is associated with a fourth confidence value;
and
generate output data indicative of one or more of the at least one first item or the at least one second item.

18 more elements

A method comprising:

accessing sensor data obtained from a plurality of weight sensors, wherein each weight sensor of the plurality of weight sensors is arranged at a different position with respect to a fixture, and the sensor data is indicative of one or more weights associated with the fixture at different positions at one or more times;
determining, based on at least a portion of the sensor data indicative of the one or more weights associated with the fixture at the different positions, a first set of event data comprising:
first event data that is indicative of a first event comprising a first weight change that occurs from a first time to a second time;
second event data that is indicative of a second event comprising a second weight change that occurs from a third time to a fourth time, wherein the third time and the fourth time occur between the first time and the second time;
and
third event data that is indicative of a third event comprising a third weight change that occurs from a fifth time to a sixth time, wherein the fifth time and the sixth time occur between the first time and the second time, and the fifth time is adjacent in time to the fourth time;
determining a first set of event hypotheses based on the second event data, wherein each event hypothesis of the first set of event hypotheses has a first confidence value;
determining a second set of event hypotheses based on the third event data, wherein each event hypothesis of the second set of event hypotheses has a second confidence value;
determining, based on the at least a portion of the sensor data and one or more of the first confidence value or the second confidence value, fourth event data that is indicative of a fourth weight change that occurs from the third time to the sixth time;
determining a third set of event hypotheses based on the fourth event data, wherein:
a first event hypothesis of the third set of event hypotheses is indicative of movement of at least one first item and is associated with a third confidence value;
and
a second event hypothesis of the third set of event hypotheses is indicative of movement of at least one second item and is associated with a fourth confidence value;
and
generating output data indicative of one or more of the at least one first item or the at least one second item.

12 more elements

Dependencies are inferred from the claim text.

Facts

Application: 15/939,080Filed 28 Mar 2018
Granted: 19 Apr 2022
Term ends: Not estimatedThe source records don’t support an estimate.

Classification13 codes

G06Q 10/087
Logistics, e.g. warehousing, loading or distribution; Inventory or stock management: inventory or stock management, e.g. order filling,…
B65G 1/1371
With arrangements or automatic control means for selecting which articles are to be removed: with data records
G01G 19/387
Weighing apparatus or methods adapted for special purposes not provided for in the preceding groups: for combinatorial weighing, i.e. selecting a…
G01G 19/42
With provisions for indicating, recording, or computing price or other quantities dependent on the weight: for counting by weighing

Family

No related US applications are recorded in the indexed filing history.

Timeline

Citations

Open elsewhere

Building an agent or pulling many patents? Use the API or MCP server instead of scraping pages. Get an API key