# US 12,217,451 B2 · Target object detection apparatus, target object detection method, and non-transitory computer-readable storage medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,217,451 B2
- Document ID: US12217451B2
- Status: Granted
- Kind: B2
- Application number: 17/770,364
- Filed: 2020-10-29
- Published: 2022-12-08
- Granted: 2025-02-04
- Earliest claimed: 2019-11-05
- Assignees: NEC Corporation
- Applicants: NEC Corporation
- Inventors: Jun Uchimura; Yuji Tahara; Rina Tomita; Yasuyo Kazo
- Web page: https://patentgrep.com/patent/US12217451B2

## CPC

- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- G06Q 10/087: Inventory or stock management, e.g. order filling, procurement or balancing against orders
- G06Q 10/08724: {by shelf level inventory management, e.g. planograms}
- G06Q 30/06: Buying, selling or leasing transactions
- G06T 7/62: of area, perimeter, diameter or volume
- G06Q 10/0877: {by inventory control or reporting using inventory tracking or counting}
- G06T 2207/30232: Surveillance
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR

## Abstract

A management apparatus (100) includes: an area computation unit (102) that acquires an area of a displayable region of a display shelf in which a product can be displayed, and computes an area ratio of an area of a non-display region of the display shelf in which the product detected from a captured image is not displayed to the area of the display shelf; a decision unit (104) that decides whether to make a notification according to the area ratio; and a notification unit (106) that makes the notification when it is decided that the notification is to be made.

## Claims

### Claim 1

A management apparatus comprising:
at least one memory storing instructions; and
at least one processor configured to execute the instructions to:
receive a captured image of a display shelf including a displayable region in which products are to be displayed;
detect the products in the displayable region;
determine, in the displayable region of the display shelf, a non-display region of the display shelf in which no products have been detected, a plurality of shapes of notification target region in the displayable region being stored with a priority order;
determine a shape of the non-display region;
compare the shape of the non-display region to shapes of a plurality of notification target regions in order of priority of the notification target regions until a matching notification target region is identified for which the shape has a degree of coincidence with the shape of the non-display region greater than a predetermined threshold,
wherein once the matching notification target region is identified, comparison of the shape of the non-display region to the shapes of other of the notification target regions that are lower in priority than the matching notification target region is bypassed; and
output a notification when the matching notification target region has been identified.

### Claim 2 (depends on 1)

The management apparatus according to claim 1, wherein the at least one processor is further configured to execute the instructions to:
determine a position of the non-display region from the captured image; and
in a case where no products are displayed in a specific position of the display shelf, as determined based on the position of the non-display region, do not perform comparison of the shape of the non-display region to the shapes of the plurality of notification target regions, and output the notification.

### Claim 3 (depends on 1)

The management apparatus according to claim 1, wherein the at least one processor is further configured to execute the instructions to:
determine a position of the non-display region from the captured image;
determine a product displayed in the non-display region of the display shelf, based on the position of the non-display region in the display shelf; and
display character information and an image related to the determined product displayed in the non-display region.

### Claim 4 (depends on 1)

The management apparatus according to claim 1, wherein the at least one processor is further configured to execute the instructions to:
determine a position of the non-display region from the captured image;
determine a product displayed in the non-display region of the display shelf, based on the position of the non-display region in the display shelf; and
compute a quantity of the determined product displayed in the non-display region, based on an area of the non-display region.

### Claim 5 (depends on 1)

The management apparatus according to claim 1, wherein the at least one processor is further configured to execute the instructions to:
as the priority order is higher, perform comparison of the shape of the non-display region to the shapes of the plurality of notification target regions in a shorter processing cycle or set a level of the notification higher; and
perform the notification according to a different notification method depending on the level of the notification.

### Claim 6

A management method performed by a management apparatus and comprising:
receiving a captured image of a display shelf including a displayable region in which products are to be displayed;
detecting the products in the displayable region;
determining, in the displayable region of the display shelf, a non-display region of the display shelf in which no products have been detected, a plurality of shapes of notification target region in the displayable region being stored with a priority order;
determining a shape of the non-display region;
comparing the shape of the non-display region to shapes of a plurality of notification target regions in order of priority of the notification target regions until a matching notification target region is identified for which the shape has a degree of coincidence with the shape of the non-display region greater than a predetermined threshold,
wherein once the matching notification target region is identified, comparison of the shape of the non-display region to the shapes of other of the notification target regions that are lower in priority than the matching notification target region is bypassed; and
outputting a notification when the matching notification target region has been identified.

### Claim 7 (depends on 6)

The management method according to claim 6, further comprising:
determining a position of the non-display region from the captured image; and
in a case where no products are displayed in a specific position of the display shelf, as determined based on the position of the non-display region, not performing comparison of the shape of the non-display region to the shapes of the plurality of notification target regions, and output the notification.

### Claim 8 (depends on 6)

The management method according to claim 6, further comprising:
determining a position of the non-display region from the captured image;
determining a product displayed in the non-display region of the display shelf, based on the position of the non-display region in the display shelf; and
displaying character information and an image related to the determined product displayed in the non-display region.

### Claim 9 (depends on 6)

The management method according to claim 6, further comprising:
determining a position of the non-display region from the captured image;
determining a product displayed in the non-display region of the display shelf, based on the position of the non-display region in the display shelf; and
computing a quantity of the determined product displayed in the non-display region, based on an area of the non-display region.

### Claim 10 (depends on 6)

The management method according to claim 6, further comprising:
as the priority order is higher, performing comparison of the shape of the non-display region to the shapes of the plurality of notification target regions in a shorter processing cycle or setting a level of the notification higher; and
performing the notification according to a different notification method depending on the level of the notification.

### Claim 11

A non-transitory computer-readable storage medium storing a program for executable by a computer to perform processing comprising:
receiving a captured image of a display shelf including a displayable region in which products are to be displayed;
detecting the products in the displayable region;
determining, in the displayable region of the display shelf, a non-display region of the display shelf in which no products have been detected, a plurality of shapes of notification target region in the displayable region being stored with a priority order;
determining a shape of the non-display region;
comparing the shape of the non-display region to shapes of a plurality of notification target regions in order of priority of the notification target regions until a matching notification target region is identified for which the shape has a degree of coincidence with the shape of the non-display region greater than a predetermined threshold,
wherein once the matching notification target region is identified, comparison of the shape of the non-display region to the shapes of other of the notification target regions that are lower in priority than the matching notification target region is bypassed; and
outputting a notification when the matching notification target region has been identified.

### Claim 12 (depends on 11)

The non-transitory computer-readable storage medium according to claim 11, wherein the processing further comprises:
determining a position of the non-display region from the captured image; and
in a case where no products are displayed in a specific position of the display shelf, as determined based on the position of the non-display region, not performing comparison of the shape of the non-display region to the shapes of the plurality of notification target regions, and output the notification.

### Claim 13 (depends on 11)

The non-transitory computer-readable storage medium according to claim 11, wherein the processing further comprises:
determining a position of the non-display region from the captured image;
determining a product displayed in the non-display region of the display shelf, based on the position of the non-display region in the display shelf; and
displaying character information and an image related to the determined product displayed in the non-display region.

### Claim 14 (depends on 11)

The non-transitory computer-readable storage medium according to claim 11, wherein the processing further comprises:
as the priority order is higher, performing comparison of the shape of the non-display region to the shapes of the plurality of notification target regions in a shorter processing cycle or setting a level of the notification higher; and
performing the notification according to a different notification method depending on the level of the notification.