# US 2023/0274402 A1 · Image processing apparatus, image capturing apparatus, image processing method, and storage medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0274402 A1
- Document ID: US20230274402A1
- Status: Published application
- Kind: A1
- Application number: 18/172,140
- Filed: 2023-02-21
- Published: 2023-08-31
- Earliest claimed: 2022-02-25
- Assignees: Canon Kabushiki Kaisha
- Applicants: Canon Kabushiki Kaisha
- Inventors: Hironobu Arifuku
- Web page: https://patentgrep.com/patent/US20230274402A1

## CPC

- G06T 3/4038: Image mosaicing, e.g. composing plane images from plane sub-images
- G06T 5/50: using two or more images, e.g. averaging or subtraction
- G06T 5/90: Dynamic range modification of images or parts thereof
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06V 10/60: relating to illumination properties, e.g. using a reflectance or lighting model
- G06V 10/761: {Proximity, similarity or dissimilarity measures}
- H04N 23/60: Control of cameras or camera modules
- G06T 2207/10144: Varying exposure
- G06T 2207/10148: Varying focus
- G06V 10/255: {Detecting or recognising potential candidate objects based on visual cues, e.g. shapes}
- G06V 20/70: Labelling scene content, e.g. deriving syntactic or semantic representations
- G06V 2201/07: Target detection
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

There is provided an image processing apparatus. An obtainment unit obtains a first image, first subject information indicating a first subject detected from the first image, a second image, and second subject information indicating a second subject detected from the second image. A composition unit generates a composite image by compositing the first image and the second image. A recording unit records one of the first subject information and the second subject information in association with the composite image in a case where a similarity between the first subject information and the second subject information satisfies a predetermined criterion, and records both of the first subject information and the second subject information in association with the composite image in a case where the similarity between the first subject information and the second subject information does not satisfy the predetermined criterion.

## Claims

### Claim 1

An image processing apparatus, comprising:
one or more memories; and
one or more processors that are in communication with the one or more memories, wherein the one or more processors and the one or more memories are configured to:
obtain a first image, first subject information indicating a first subject detected from the first image, a second image, and second subject information indicating a second subject detected from the second image;
generate a composite image by compositing the first image and the second image; and
record one of the first subject information and the second subject information in association with the composite image in a case where a similarity between the first subject information and the second subject information satisfies a predetermined criterion, and record both of the first subject information and the second subject information in association with the composite image in a case where the similarity between the first subject information and the second subject information does not satisfy the predetermined criterion.

### Claim 2 (depends on 1)

The image processing apparatus according to claim 1, wherein
the similarity between the first subject information and the second subject information satisfies the predetermined criterion in a case where a degree of overlap between a region of the first subject specified by the first subject information and a region of the second subject specified by the second subject information is equal to or higher than a predetermined degree.

### Claim 3 (depends on 1)

The image processing apparatus according to claim 1, wherein, in a case where the similarity between the first subject information and the second subject information satisfies the predetermined criterion, the one or more processors and the one or more memories are further configured to
record the first subject information in association with the composite image in a case where the composite image includes more components derived from the first subject than components derived from the second subject, and
record the second subject information in association with the composite image in a case where the composite image includes more components derived from the second subject than components derived from the first subject.

### Claim 4 (depends on 1)

The image processing apparatus according to claim 1, wherein
the first subject information includes information indicating a type of the first subject,
the second subject information includes information indicating a type of the second subject, and
in a case where the similarity between the first subject information and the second subject information satisfies the predetermined criterion, the one or more processors and the one or more memories are further configured to
record the first subject information in association with the composite image in a case where the information indicating the type of the first subject is more detailed than the information indicating the type of the second subject, and
record the second subject information in association with the composite image in a case where the information indicating the type of the second subject is more detailed than the information indicating the type of the first subject.

### Claim 5 (depends on 1)

The image processing apparatus according to claim 1, wherein
the first image and the second image are images that have been shot under different exposure settings, and
the one or more processors and the one or more memories are further configured to composite the first image and the second image so as to expand a luminance dynamic range.

### Claim 6 (depends on 5)

The image processing apparatus according to claim 5, wherein
in a case where the first image and the second image have been shot in a predetermined shooting mode, the one or more processors and the one or more memories are further configured to record both of the first subject information and the second subject information in association with the composite image regardless of whether the similarity between the first subject information and the second subject information satisfies the predetermined criterion.

### Claim 7 (depends on 6)

The image processing apparatus according to claim 6, wherein
the predetermined shooting mode is a night view mode.

### Claim 8 (depends on 1)

The image processing apparatus according to claim 1, wherein
the first image and the second image are images that have been shot under different focusing distance settings, and
the one or more processors and the one or more memories are further configured to composite the first image and the second image so as to increase a depth of field.

### Claim 9 (depends on 1)

The image processing apparatus according to claim 1, wherein the one or more processors and the one or more memories are further configured to
detect the first subject from the first image, and detect the second subject from the second image;
generate the first subject information indicating the first subject detected from the first image, and generate the second subject information indicating the second subject detected from the second image.

### Claim 10 (depends on 9)

The image processing apparatus according to claim 9, wherein the one or more processors and the one or more memories are further configured to
generate the first subject information and the second subject information by executing inference processing using an inference model with respect to the first subject detected from the first image and the second subject detected from the second image.

### Claim 11 (depends on 1)

An image capturing apparatus, comprising:
the image processing apparatus according to claim 1; and
an image capturing unit configured to generate the first image and the second image,
wherein the obtainment unit obtains the first image and the second image generated by the image capturing unit.

### Claim 12

An image processing method executed by an image processing apparatus, the method comprising:
obtaining a first image, first subject information indicating a first subject detected from the first image, a second image, and second subject information indicating a second subject detected from the second image;
generating a composite image by compositing the first image and the second image; and
recording one of the first subject information and the second subject information in association with the composite image in a case where a similarity between the first subject information and the second subject information satisfies a predetermined criterion, and recording both of the first subject information and the second subject information in association with the composite image in a case where the similarity between the first subject information and the second subject information does not satisfy the predetermined criterion.

### Claim 13

A non-transitory computer-readable storage medium which stores computer-executable instructions for causing a computer to execute an image processing method comprising:
obtaining a first image, first subject information indicating a first subject detected from the first image, a second image, and second subject information indicating a second subject detected from the second image;
generating a composite image by compositing the first image and the second image; and
recording one of the first subject information and the second subject information in association with the composite image in a case where a similarity between the first subject information and the second subject information satisfies a predetermined criterion, and recording both of the first subject information and the second subject information in association with the composite image in a case where the similarity between the first subject information and the second subject information does not satisfy the predetermined criterion.
