On-shelf image based out-of-stock detection

A non-transitory, computer-readable medium comprising instructions that, when executed by a processor, cause the processor to:

• receive first image data of a product display area in a store from a camera, the product display area including a plurality of product labels;
• determine, from the image data, a position of each of the plurality product labels within the product display area;
• generate, for each product label, a bounding box within the product display area based on the position of each product label within the product display area, each bounding box including a respective plurality of products;
• identify a product associated with each product label;
• associate, for each product label, the identified product with the bounding box for the product label;
• and
• perform an out-of-stock detection process that comprises:
• receiving second image data of the product display area from the camera, the second image data being captured a different time relative to the first image data corresponding to image data for each bounding box identified using the first image data, each bounding box including its respective plurality of products;
• providing an extracted area corresponding to each bounding box as input to a classifier model, the extracted area corresponding to at least two of the respective plurality of products;
• receiving, as output from the classifier model, an indication that a given product in a given portion of the second image data corresponding to a given bounding box is out-of-stock, wherein the classifier model is a machine learning model that is trained to output that the given product is out of stock where one or more voids are detected within a bounding box while the bounding box features at least two of the given product;
• and
• sending a restocking notification for the identified product to a store client device responsive to receiving the indication.

A non-transitory, computer-readable medium comprising instructions that, when executed by a processor, cause the processor to:

• obtain, at a first time, first image data of a product display area in a store at a first camera resolution, the product display area including a plurality of product labels;
• generate a bounding box for each of the plurality of product labels of the product display area based on a position of each product label within the product display area each bounding box including a respective plurality of products;
• identify a product associated with each product label;
• store, for each product label, information the identified product with the bounding box for the product label;
• receive, after the first time, second image data of the product display area at a second camera resolution, the second image data corresponding to image data for each bounding box identified using the first image data, the first camera resolution being higher relative to the second camera resolution;
• provide a portion of the second image data corresponding to each bounding box as input to a classifier model;
• receive, as output from the classifier model, an indication that a given product in a given portion of the second image data corresponding to a given bounding box is out-of- stock, wherein the classifier model is a machine learning model that is trained to output that the given product is out-of-stock where the bounding box is less than full and features at least [[one]] two of the given product;
• and
• send a restocking notification for the identified product to a store client device.