Detection of object removal and replacement from a shelf

A method, comprising:

• determining that a person has interacted with a rack comprising shelves configured to store items;
• receive an image feed comprising frames of angled-view images generated by an image sensor after the person has interacted with the rack, wherein the image sensor is positioned such that a field-of-view of the image sensor encompasses at least a portion of the rack, wherein the image sensor is configured to generate the angled-view images of the items stored on the shelves of the rack;
• determining, based on at least one angled-view image of the image feed, that the person interacted with a first item stored on the rack;
• selecting a first image from the image feed associated with a first time before the person interacted with the first item, and a second image from the image feed associated with a second time after the person interacted with the first item;
• over a period of time, tracking a pixel position of a wrist of the person in the image feed;
• determining, based on the tracked pixel position of the wrist, a region-of interest defining a portion of the first image and a portion of the second image;
• determining, based on a comparison of the portion of the first image defined by the region-of-interest to the portion of the second image defined by the region-of-interest, whether the first item was removed from the rack or the item was placed on the rack;
• if it is determined that the first item was removed from the rack, assigning the first item to the person;
• and
• if it is determined that the first item was placed on the rack, unassign the first item from the person.

A tracking subsystem comprising at least one processor configured to:

• determine that a person has interacted with a rack comprising shelves configured to store items;
• receive an image feed comprising frames of angled-view images generated by an image sensor after the person has interacted with the rack, wherein the image sensor is positioned such that a field-of-view of the image sensor encompasses at least a portion of the rack, wherein the image sensor is configured to generate the angled-view images of the items stored on the shelves of the rack;
• determine, based on at least one angled-view image of the image feed, that the person interacted with a first item stored on the rack;
• select a first image from the image feed associated with a first time before the person interacted with the first item, and a second image from the image feed associated with a second time after the person interacted with the first item;
• over a period of time, track a pixel position of a wrist of the person in the image feed;
• determine, based on the tracked pixel position of the wrist, a region-of interest defining a portion of the first image and a portion of the second image;
• determine, based on a comparison of the portion of the first image defined by the region-of-interest to the portion of the second image defined by the region-of-interest, whether the first item was removed from the rack or the item was placed on the rack;
• if it is determined that the first item was removed from the rack, assign the first item to the person;
• and
• if it is determined that the first item was placed on the rack, unassign the first item from the person.