# US 2023/0276117 A1 · Main object determination apparatus, imaging apparatus, and control method for controlling main object determination apparatus

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0276117 A1
- Document ID: US20230276117A1
- Status: Published application
- Kind: A1
- Application number: 18/172,598
- Filed: 2023-02-22
- Published: 2023-08-31
- Earliest claimed: 2022-02-25
- Assignees: Canon Kabushiki Kaisha
- Applicants: Canon Kabushiki Kaisha
- Inventors: TOMOHIRO NISHIYAMA; REIJI HASEGAWA
- Web page: https://patentgrep.com/patent/US20230276117A1

## CPC

- G06V 10/62: relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking
- H04N 23/611: where the recognised objects include parts of the human body
- G06V 10/40: Extraction of image or video features
- G06V 10/757: {Matching configurations of points or features}
- G06V 10/945: {User interactive design; Environments; Toolboxes}
- G06V 40/10: Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V 40/23: {Recognition of whole body movements, e.g. for sport training}
- H04N 23/631: {Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters}
- H04N 23/6811: {based on the image signal}
- H04N 23/695: Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
- H04N 23/72: Combination of two or more compensation controls
- H04N 23/617: Upgrading or updating of programs or applications for camera control
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A main object determination apparatus includes an image acquisition unit configured to acquire images captured at different timings, a selection unit configured to select main object candidate(s) from objects in the images, a determination unit configured to determine whether the main object candidate(s) each selected at the respective different timings are the same, and an input unit configured to receive an operation. In a case where the determination unit determines that the main object candidate(s) selected by the selection unit in an image of interest and one or more images captured within a predetermined time before the image of interest is captured are the same, the determination unit determines the main object candidate(s) to be a main object. In a case where the input unit receives an instruction to specify a new main object, the determination unit switches the main object according to the instruction.

## Claims

### Claim 1

A main object determination apparatus comprising:
one or more processors; and
a memory storing instructions which, when executed by the one or more processors, cause the main object determination apparatus to function as:
an image acquisition unit configured to acquire images captured at different timings;
a selection unit configured to select, using information regarding feature points of objects in the images, one or more main object candidates from the objects;
a determination unit configured to determine, using information regarding feature amounts calculated from the feature points, whether the one or more main object candidates each selected at the respective different timings are the same; and
an input unit configured to receive an operation from a user,
wherein, in a case where the determination unit determines that the one or more main object candidates selected by the selection unit in an image of interest and one or more images captured within a predetermined time before the image of interest is captured are the same, the determination unit determines the one or more main object candidates to be a main object, and
wherein, in a case where the input unit receives an instruction to specify a new main object from the user, the determination unit switches the main object according to the instruction.

### Claim 2 (depends on 1)

The main object determination apparatus according to claim 1, wherein each of the objects represents a person or an animal.

### Claim 3 (depends on 1)

The main object determination apparatus according to claim 1, wherein the feature points include at least one or more joint positions of the respective objects.

### Claim 4 (depends on 1)

The main object determination apparatus according to claim 1, wherein the information about the feature amounts includes a center of gravity of the respective objects, position information indicating a part of a body of the respective objects, and a position or a motion vector of the respective objects calculated from the feature points.

### Claim 5

A main object determination apparatus comprising:
one or more processors; and
a memory storing instructions which, when executed by the one or more processors, cause the main object determination apparatus to function as:
an image acquisition unit configured to acquire images captured at different timings;
a selection unit configured to select one or more main object candidates from objects in the images;
a determination unit configured to determine whether the one or more main object candidates each selected at the respective different timings are the same; and
an input unit configured to receive an operation from a user,
wherein the selection unit selects the one or more main object candidates in one or more images captured within a predetermined time before an image of interest is captured, and in a case where the determination unit determines that the one or more main object candidates in the one or more images captured within the predetermined time are the same as a main object candidate in the image of interest, the determination unit determines the candidates to be a main object, and
wherein, in a case where the input unit receives an instruction to specify a new main object from the user, the determination unit switches the main object according to the instruction.

### Claim 6 (depends on 1)

The main object determination apparatus according to claim 1, wherein the selection unit calculates a reliability corresponding to a degree of possibility of being a main object from each of the objects.

### Claim 7 (depends on 6)

The main object determination apparatus according to claim 6, wherein the selection unit calculates the reliability using a distance between the respective objects and an important physical body.

### Claim 8 (depends on 6)

The main object determination apparatus according to claim 6, wherein the selection unit selects an object of which the reliability has the greatest value as a main object candidate.

### Claim 9 (depends on 8)

The main object determination apparatus according to claim 8, wherein the selection unit selects also an object in which a difference from the greatest value of the reliability has a value smaller than a value determined in advance as a main object candidate.

### Claim 10 (depends on 1)

The main object determination apparatus according to claim 1, wherein the selection unit does not perform selection of the one or more main object candidates from an image that is not captured within the predetermined time before the image of interest is captured.

### Claim 11 (depends on 1)

The main object determination apparatus according to claim 1, further comprising a tracking unit configured to track the objects,
wherein, in a case where the determination unit determines that the one or more main object candidates are the same, the tracking unit switches a tracking target in the image of interest to the main object.

### Claim 12 (depends on 1)

The main object determination apparatus according to claim 1, wherein, in a case where an input for specifying a main object is provided to the input unit, the main object specified by the input is given priority over the main object determined by the selection unit and the determination unit.

### Claim 13 (depends on 12)

The main object determination apparatus according to claim 12, wherein a cancellation input for cancelling the main object determined by the selection unit and the determination unit is provided to the input unit, a main object in an image captured immediately before the cancellation input is provided or an object before the tracking target is switched is determined to be a main object.

### Claim 14 (depends on 11)

The main object determination apparatus according to claim 11, further comprising a display unit configured to display the images,
wherein the display unit displays an image obtained by performing superimposed display of at least one of the one or more main object candidates detected in the images, the main object, and the tracking target that is being tracked.

### Claim 15 (depends on 14)

The main object determination apparatus according to claim 14, wherein the display unit changes a color, thickness, and a shape of the superimposed display.

### Claim 16 (depends on 14)

The main object determination apparatus according to claim 14, wherein the display unit differentiates the superimposed display between the one or more main object candidates in the images, the main object, and the tracking target.

### Claim 17 (depends on 1)

An imaging apparatus comprising:
an imaging unit configured to capture an object image formed through an imaging optical system; and
the main object determination apparatus according to claim 1.

### Claim 18

A control method for controlling a main object determination apparatus, the control method comprising:
acquiring images captured at different timings;
selecting, using information about feature points of objects in the images, one or more main object candidates from the objects;
determining, using information regarding feature amounts calculated from the feature points, whether the one or more main object candidates selected at the respective different timings are the same; and
receiving an operation from a user,
wherein, in a case where it is determined that the one or more main object candidates selected in an image of interest and one or more images captured within a predetermined time before the image of interest is captured are the same, the one or more main object candidates are determined to be a main object, and
wherein, in a case where an instruction to specify a new main object is received from the user, the main object is switched according to the instruction.

### Claim 19

A control method for controlling a main object determination apparatus, the control method comprising:
acquiring images captured at different timings;
selecting one or more main object candidates from objects in the images;
determining whether the one or more main object candidates selected at the respective different timings are the same; and
receiving an operation from a user,
wherein the one or more main object candidates are selected in one or more images captured within a predetermined time before an image of interest is captured,
wherein, in a case where it is determined that the one or more main object candidates in the images captured within the predetermined time are the same as a main object candidate in the image of interest, the candidates are determined to be a main object, and
wherein, in a case where an instruction to specify a new main object is received from the user, the main object is switched according to the instruction.

### Claim 20 (depends on 18)

A non-transitory computer-readable storage medium storing a program for causing a computer to perform the control method according to claim 18.

### Claim 21 (depends on 19)

A non-transitory computer-readable storage medium storing a program for causing a computer to perform the control method according to claim 19.
