Non-product detection in smart shopping carts
AssigneeAmazon Technologies, Inc.
InventorsWeijian Li, Yin Wang, Shun Miao
Life of this patent
Estimating the term…
Abstract
This disclosure describes, in part, systems and methods enabling non-product detection using a dual-pathway model. Images can be collected of non-product items, such as by using a shopping cart equipped with a basket facing camera or other mobile apparatus. A registration status of a user can be determined. The product images can be analyzed to determine whether the non-product item matches a registered non-product item. The product images can be analyzed to determine whether the non-product item matches a category of a group of non-product items for a particular store.
Claims19
A method to detect non-product items in a shopping cart of a store, comprising:
- receiving a first sequence of images of an item in response to an event during a session associated with a user identifier, the first sequence of images being generated by sensors of the shopping cart, wherein a virtual cart is generated in associated with the session;
- determining a registration status associated with the user identifier;
- based on determining that a personal item is registered in association with the user identifier, determining whether the first sequence of images matches a second sequence of images of a registered non-product item, wherein the determining comprises:
- accessing the second sequence of images of the registered non-product item;
- determining similarity of the first sequence of images to the second sequence of images, wherein the similarity is based on generating respective similarity scores for each respective image in the first sequence of images and respective features for each of the images in the second sequence of images;
- determining, based on the respective similarity scores, that the first sequence of images matches the registered non-product item;
- and
- storing an indication that the item matches the registered non-product item;
- based on determining that the personal item is not registered in association with the user identifier, determining, whether the item matches a non-product category of a group of non-product categories for the store, wherein the determining comprises:
- retrieving the group of non-product categories of the non-product items;
- determining, for each of the non-product categories, a respective matching image of the first sequence of images of the item, wherein the determining generates respective scores for each of the images in the first sequence of images and the respective non-product categories;
- and
- determining, based on the respective scores, that the item matches one of the non-product categories;
- and
- performing, during the session or at an end of the session, an action.
A method to detect non-product items in a shopping cart using a group of non-product categories, comprising:
- receiving an image of an item, the image being generated by sensors of the shopping cart;
- determining, for each category of the group of non-product categories, a similarity score for each category and the image;
- determining, based on the respective similarity scores, that the item matches one of the categories of the group of the non-product categories by at least:
- determining a representative similarity score of the respective similarity scores, the representative similarity score comprising one of a global similarity score, a mean similarity score, or a maximum similarity score;
- comparing the representative similarity score to a threshold similarity score;
- and
- determining the representative similarity score is greater than the threshold similarity score;
- and
- performing an action.
A method for detecting a non-product item in a shopping cart, comprising:
- receiving a first sequence of images associated with an item, the first sequence of images being generated by sensors of a shopping cart;
- determining a personal item is registered as a registered non-product item in association with a user identifier;
- accessing a second sequence of images of the registered non-product item;
- determining similarity scores of the first sequence of images and the second sequence of images by at least:
- determining a representative similarity score of the similarity scores, the representative similarity score comprising one of a global similarity score, a mean similarity score, or a maximum similarity score;
- comparing the representative similarity score to a threshold similarity score;
- and
- determining the representative similarity score is greater than the threshold similarity score;
- determining, based on the similarity scores, that the item matches the registered non-product item;
- and
- performing an action.
Dependencies are inferred from the claim text.