# US 2025/0156807 A1 · On-shelf image based out-of-stock detection

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0156807 A1
- Document ID: US20250156807A1
- Status: Published application
- Kind: A1
- Application number: 19/024,986
- Filed: 2025-01-16
- Published: 2025-05-15
- Earliest claimed: 2018-06-29
- Assignees: Focal Systems, Inc.
- Applicants: Focal Systems, Inc.
- Inventors: Francois Chaubard
- Web page: https://patentgrep.com/patent/US20250156807A1

## CPC

- G06Q 10/087: Inventory or stock management, e.g. order filling, procurement or balancing against orders
- H04N 23/57: Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
- G06Q 10/0877: {by inventory control or reporting using inventory tracking or counting}
- G06T 11/23: using straight lines or curves
- G06V 30/1473: {Recognising objects as potential recognition candidates based on visual cues, e.g. shapes}
- H04N 23/54: Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
- H04N 23/698: for achieving an enlarged field of view, e.g. panoramic image capture
- G06T 2207/20076: Probabilistic image processing
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30232: Surveillance
- G06T 2210/12: Bounding box
- G06V 30/10: Character recognition
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

An out-of-stock detection system notifies store management that a product is out of stock. The system captures images of a shelf and determines the position product labels thereon. For each product label, a bounding box is generated based on the position of each product label on the shelf. The system then identifies a product for each product label based on information within each product label and, for each product label, stores a product identified for each bounding box. Accordingly, the system performs an out-of-stock detection process that includes capturing additional image data of the shelf periodically that includes each bounding box, providing a portion of the additional image data for each bounding box to a model trained to determine whether the bounding box contains products, sending a notification for a product determined to be out of stock to a store client device based on output from the model.

## Claims

### Claim 1

A method for performing a restock detection process, the method comprising:
capturing, using a camera mounted in view of a shelf unit, a deployment-stage image of a product display area of the shelf unit, the deployment-stage image including at least a product label describing a product and a portion of the product display area for stocking the product;
accessing a template image of the product display area, the template image including a product-level bounding box corresponding to the product label, the product-level bounding box in the template image representing a template number of individual products corresponding to the product label in the portion of the product display area for stocking the product;
generating, based on the deployment-stage image and within the portion of the product display area that corresponds to the product-level bounding box, individual bounding boxes for each identified individual product within a display location corresponding to the individual bounding box;
comparing a number of individual products corresponding to the product label identified based on the generated individual bounding boxes with the template number of individual products; and
transmitting a notification of a restock state for the product label based on the comparison.

### Claim 2 (depends on 1)

The method of claim 1, wherein the template image is an image of the product display area where the portion of the product display area for stocking the product is fully stocked with the product.

### Claim 3 (depends on 1)

The method of claim 1, wherein the template image includes a second product-level bounding box corresponding to a second product label, the second product-level bounding box in the template image representing a template number of individual second products corresponding to the second product label in a second portion of the product display area for stocking a second product, the method further comprising:
generating, based on the deployment-stage image and within the second portion of the product display area that corresponds to the second product-level bounding box for the second product, individual bounding boxes for each identified individual second product within a display location corresponding to the individual bounding box.

### Claim 4 (depends on 1)

The method of claim 1, further comprising:
determining a historical rate at which the product is removed from the shelf unit, wherein the notification of the restock state for the product label is sent further based on the determined historical rate.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
determining that the product on the shelf unit is in a fully stocked state in response to determining that the number of individual products corresponding to the product label identified based on the generated individual bounding boxes is same as, or within a predefined threshold of, the template number of individual products.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
capturing, during an onboarding process, the template image of the product display area of the shelf unit;
determining a position of the product label within the product display area;
identifying the product associated with the product label based on product information included in the product label;
generating the product-level bounding box within the product display area based on the position of the product label within the product display area and the identified product; and
storing an association of the identified product with the product-level bounding box for the product label.

### Claim 7 (depends on 6)

The method of claim 6, wherein identifying the product associated with the product label comprises:
extracting the product information from the product label by applying an optical character recognition algorithm to the template image; and
matching the product information to a database of products carried by a store.

### Claim 8

A non-transitory computer-readable medium comprising memory with instructions encoded thereon for performing a restock detection process, the instructions when executed causing at least one processor to perform operations, the instructions comprising instructions to:
capture, using a camera mounted in view of a shelf unit, a deployment-stage image of a product display area of the shelf unit, the deployment-stage image including at least a product label describing a product and a portion of the product display area for stocking the product;
access a template image of the product display area, the template image including a product-level bounding box corresponding to the product label, the product-level bounding box in the template image representing a template number of individual products corresponding to the product label in the portion of the product display area for stocking the product;
generate, based on the deployment-stage image and within the portion of the product display area that corresponds to the product-level bounding box, individual bounding boxes for each identified individual product within a display location corresponding to the individual bounding box;
compare a number of individual products corresponding to the product label identified based on the generated individual bounding boxes with the template number of individual products; and
transmit a notification of a restock state for the product label based on the comparison.

### Claim 9 (depends on 8)

The non-transitory computer-readable medium of claim 8, wherein the template image is an image of the product display area where the portion of the product display area for stocking the product is fully stocked with the product.

### Claim 10 (depends on 8)

The non-transitory computer-readable medium of claim 8, wherein the template image includes a second product-level bounding box corresponding to a second product label, the second product-level bounding box in the template image representing a template number of individual second products corresponding to the second product label in a second portion of the product display area for stocking a second product, the instructions further comprising instructions to:
generate, based on the deployment-stage image and within the second portion of the product display area that corresponds to the second product-level bounding box for the second product, individual bounding boxes for each identified individual second product within a display location corresponding to the individual bounding box.

### Claim 11 (depends on 8)

The non-transitory computer-readable medium of claim 8, the instructions further comprising instructions to:
determine a historical rate at which the product is removed from the shelf unit, wherein the notification of the restock state for the product label is sent further based on the determined historical rate.

### Claim 12 (depends on 8)

The non-transitory computer-readable medium of claim 8, the instructions further comprising instructions to:
determine that the product on the shelf unit is in a fully stocked state in response to determining that the number of individual products corresponding to the product label identified based on the generated individual bounding boxes is same as, or within a predefined threshold of, the template number of individual products.

### Claim 13 (depends on 8)

The non-transitory computer-readable medium of claim 8, the instructions further comprising instructions to:
capture, during an onboarding process, the template image of the product display area of the shelf unit;
determine a position of the product label within the product display area;
identify the product associated with the product label based on product information included in the product label;
generate the product-level bounding box within the product display area based on the position of the product label within the product display area and the identified product; and
store an association of the identified product with the product-level bounding box for the product label.

### Claim 14 (depends on 13)

The non-transitory computer-readable medium of claim 13, wherein the instructions to identify the product associated with the product label comprise instructions to:
extract the product information from the product label by applying an optical character recognition algorithm to the template image; and
match the product information to a database of products carried by a store.

### Claim 15

A system comprising:
memory with instructions encoded thereon for performing a restock detection process; and
one or more processors that, when executing the instructions, are caused to perform operations comprising:
capturing, using a camera mounted in view of a shelf unit, a deployment-stage image of a product display area of the shelf unit, the deployment-stage image including at least a product label describing a product and a portion of the product display area for stocking the product;
accessing a template image of the product display area, the template image including a product-level bounding box corresponding to the product label, the product-level bounding box in the template image representing a template number of individual products corresponding to the product label in the portion of the product display area for stocking the product;
generating, based on the deployment-stage image and within the portion of the product display area that corresponds to the product-level bounding box, individual bounding boxes for each identified individual product within a display location corresponding to the individual bounding box;
comparing a number of individual products corresponding to the product label identified based on the generated individual bounding boxes with the template number of individual products; and
transmitting a notification of a restock state for the product label based on the comparison.

### Claim 16 (depends on 15)

The system of claim 15, wherein the template image is an image of the product display area where the portion of the product display area for stocking the product is fully stocked with the product.

### Claim 17 (depends on 15)

The system of claim 15, wherein the template image includes a second product-level bounding box corresponding to a second product label, the second product-level bounding box in the template image representing a template number of individual second products corresponding to the second product label in a second portion of the product display area for stocking a second product, the operations further comprising:
generating, based on the deployment-stage image and within the second portion of the product display area that corresponds to the second product-level bounding box for the second product, individual bounding boxes for each identified individual second product within a display location corresponding to the individual bounding box.

### Claim 18 (depends on 15)

The system of claim 15, the operations further comprising:
determining a historical rate at which the product is removed from the shelf unit, wherein the notification of the restock state for the product label is sent further based on the determined historical rate.

### Claim 19 (depends on 15)

The system of claim 15, the operations further comprising:
determining that the product on the shelf unit is in a fully stocked state in response to determining that the number of individual products corresponding to the product label identified based on the generated individual bounding boxes is same as, or within a predefined threshold of, the template number of individual products.

### Claim 20 (depends on 15)

The system of claim 15, the operations further comprising:
capturing, during an onboarding process, the template image of the product display area of the shelf unit;
determining a position of the product label within the product display area;
identifying the product associated with the product label based on product information included in the product label;
generating the product-level bounding box within the product display area based on the position of the product label within the product display area and the identified product; and
storing an association of the identified product with the product-level bounding box for the product label.
