# US 2023/0410321 A1 · Information processing apparatus, control method, and program

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0410321 A1
- Document ID: US20230410321A1
- Status: Published application
- Kind: A1
- Application number: 18/227,850
- Filed: 2023-07-28
- Published: 2023-12-21
- Earliest claimed: 2019-12-17
- Assignees: NEC Corporation
- Applicants: NEC Corporation
- Inventors: Soma SHIRAISHI
- Web page: https://patentgrep.com/patent/US20230410321A1

## CPC

- G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51)
- G06Q 30/02: Marketing; Price estimation or determination; Fundraising
- G06T 7/248: {involving reference images or patches}
- G06T 7/73: using feature-based methods
- G06T 7/74: {involving reference images or patches}
- G06V 10/255: {Detecting or recognising potential candidate objects based on visual cues, e.g. shapes}
- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- G06V 40/107: {Static hand or arm}
- G06V 40/28: {Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113)}
- G08B 13/196: using television cameras
- G06T 2207/10016: Video; Image sequence
- G06T 2207/10021: Stereoscopic video; Stereoscopic image sequence
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2207/20081: Training; Learning
- G06T 2207/30196: Human being; Person
- G06T 2207/30232: Surveillance
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G08B: SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS

## Abstract

An information processing apparatus (2000) analyzes a captured image (12) generated by a camera (10) to determine a motion of a person. The camera (10) is a camera that images a display place where an item is displayed. The information processing apparatus (2000) detects a reference position (24) from the captured image (12). The reference position (24) indicates a position of a hand of the person. The information processing apparatus (2000) decides an analysis target region (30) to be analyzed in the captured image (12) using the reference position (24). The information processing apparatus (2000) analyzes the analysis target region (30) to determine the motion of the person.

## Claims

### Claim 1

An information processing apparatus comprising:
at least one memory configured to store instructions; and
at least one processor configured to execute the instructions to:
detect a reference position indicating a position of a hand of a person included in a captured image from the captured image in which a display place of an item is imaged;
decide an analysis target region in the captured image using the detected reference position and decides the analysis target region; and
determine a motion of the person by analyzing the decided analysis target region decided for each of a plurality of captured images generated at different timepoints.

### Claim 2 (depends on 1)

The information processing apparatus according to claim 1,
wherein the plurality of captured images includes a first captured image including a scene in which the reference position moves toward a product shelf, and a second captured image including a scene in which the reference position moves in a direction away from the product shelf from among the captured images generated later than generated timepoint of the first captured image.

### Claim 3 (depends on 2)

The information processing apparatus according to claim 2,
wherein the first captured image is imaged in a case where a distance between the reference position and the product shelf decreases over time.

### Claim 4 (depends on 2)

The information processing apparatus according to claim 2,
wherein the second captured image is imaged in a case where a distance between the reference position and the product shelf increases over time.

### Claim 5 (depends on 1)

The information processing apparatus according to claim 1,
wherein the at least one processor is further configured to execute the instructions to determine at least one of a motion of the person taking out an item from the display place, a motion of the person not holding an item both before and after contact with the display place, a motion of the person placing an item on the display place, and a motion of the person holding an item both before and after contact with the display place.

### Claim 6 (depends on 5)

The information processing apparatus according to claim 5,
wherein the at least one processor is further configured to execute the instructions to determine an item which is a target of the motion of the person based on display information indicating a position of each item in the display place and a position of the reference position in a height direction included in the analysis target region.

### Claim 7 (depends on 1)

The information processing apparatus according to claim 1,
wherein the at least one processor is further configured to execute the instructions to decide a size of the analysis target region that is decided by a reference area defined in advance for the hand region, or a reference distance value defined in advance.

### Claim 8

A control method executed by a computer, the method comprising:
detecting a reference position indicating a position of a hand of a person included in a captured image from the captured image in which a display place of an item is imaged;
deciding an analysis target region in the captured image using the detected reference position and deciding the analysis target region; and
determining a motion of the person by analyzing the decided analysis target region decided for each of a plurality of captured images generated at different timepoints.

### Claim 9 (depends on 8)

The control method according to claim 8,
wherein the plurality of captured images includes a first captured image including a scene in which the reference position moves toward a product shelf, and a second captured image including a scene in which the reference position moves in a direction away from the product shelf from among the captured images generated later than generated timepoint of the first captured image.

### Claim 10 (depends on 9)

The control method according to claim 9,
wherein the first captured image is imaged in a case where a distance between the reference position and the product shelf decreases over time.

### Claim 11 (depends on 2)

The control method according to claim 2,
wherein the second captured image is imaged in a case where a distance between the reference position and the product shelf increases over time.

### Claim 12 (depends on 8)

The control method according to claim 8,
wherein the method comprises determining at least one of a motion of the person taking out an item from the display place, a motion of the person not holding an item both before and after contact with the display place, a motion of the person placing an item on the display place, and a motion of the person holding an item both before and after contact with the display place.

### Claim 13 (depends on 12)

The control method according to claim 12,
wherein the method comprises determining an item which is a target of the motion of the person based on display information indicating a position of each item in the display place and a position of the reference position in a height direction included in the analysis target region.

### Claim 14 (depends on 8)

The control method according to claim 8,
wherein the method comprises deciding a size of the analysis target region that is decided by a reference area defined in advance for the hand region, or a reference distance value defined in advance.

### Claim 15

A non-transitory computer-readable medium storing a program for causing a computer to perform operations, the operations comprising:
detecting a reference position indicating a position of a hand of a person included in a captured image from the captured image in which a display place of an item is imaged;
deciding an analysis target region in the captured image using the detected reference position and deciding the analysis target region; and
determining a motion of the person by analyzing the decided analysis target region decided for each of a plurality of captured images generated at different timepoints.

### Claim 16 (depends on 15)

The non-transitory computer-readable medium according to claim 15,
wherein the plurality of captured images includes a first captured image including a scene in which the reference position moves toward a product shelf, and a second captured image including a scene in which the reference position moves in a direction away from the product shelf from among the captured images generated later than generated timepoint of the first captured image.

### Claim 17 (depends on 16)

The non-transitory computer-readable medium according to claim 16,
wherein the first captured image is imaged in a case where a distance between the reference position and the product shelf decreases over time.

### Claim 18 (depends on 16)

The non-transitory computer-readable medium according to claim 16,
wherein the second captured image is imaged in a case where a distance between the reference position and the product shelf increases over time.

### Claim 19 (depends on 15)

The non-transitory computer-readable medium according to claim 15,
wherein the operations comprise determining at least one of a motion of the person taking out an item from the display place, a motion of the person not holding an item both before and after contact with the display place, a motion of the person placing an item on the display place, and a motion of the person holding an item both before and after contact with the display place.

### Claim 20 (depends on 19)

The non-transitory computer-readable medium according to claim 19,
wherein the operations comprise determining an item which is a target of the motion of the person based on display information indicating a position of each item in the display place and a position of the reference position in a height direction included in the analysis target region.
