# US 2024/0005467 A1 · Image processing apparatus, imaging apparatus, image processing method, and program

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0005467 A1
- Document ID: US20240005467A1
- Status: Published application
- Kind: A1
- Application number: 18/338,225
- Filed: 2023-06-20
- Published: 2024-01-04
- Earliest claimed: 2022-06-30
- Assignees: FUJIFILM Corporation
- Applicants: FUJIFILM Corporation
- Inventors: Taro SAITO; Koichi TANAKA; Tomoharu SHIMADA
- Web page: https://patentgrep.com/patent/US20240005467A1

## CPC

- G06T 5/50: using two or more images, e.g. averaging or subtraction
- G06T 3/4053: based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
- G06T 5/60: using machine learning, e.g. neural networks
- G06T 5/70: Denoising; Smoothing
- G06T 5/80: Geometric correction
- G06T 7/13: Edge detection
- G06V 10/44: Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06V 10/56: relating to colour
- G06V 10/98: Detection or correction of errors, e.g. by rescanning the pattern or by human intervention; Evaluation of the quality of the acquired patterns
- G06V 40/10: Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/20192: Edge enhancement; Edge preservation
- G06T 2207/20221: Image fusion; Image merging
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

The image processing apparatus includes a processor. The processor is configured to: acquire a first image, which is obtained by performing first AI processing on a processing target image, and a second image, which is obtained without performing the first AI processing on the processing target image; and adjust excess and deficiency of the first AI processing by combining the first image and the second image.

## Claims

### Claim 1

An image processing apparatus comprising:
a processor,
wherein the processor is configured to:
acquire a first image, which is obtained by performing first AI processing on a processing target image, and a second image, which is obtained without performing the first AI processing on the processing target image; and
adjust excess and deficiency of the first AI processing by combining the first image and the second image.

### Claim 2 (depends on 1)

The image processing apparatus according to claim 1,
wherein the second image is an image obtained by performing non-AI method processing, which does not use a neural network, on the processing target image.

### Claim 3

An image processing apparatus comprising:
a processor,
wherein the processor is configured to:
acquire a first image, which is obtained by performing first AI processing on a processing target image to adjust a non-noise element of the processing target image, and a second image, which is obtained without performing the first AI processing on the processing target image; and
adjust the non-noise element by combining the first image and the second image.

### Claim 4 (depends on 3)

The image processing apparatus according to claim 3,
wherein the second image is an image in which the non-noise element is adjusted by performing non-AI method processing, which does not use a neural network, on the processing target image.

### Claim 5 (depends on 3)

The image processing apparatus according to claim 3,
wherein the second image is an image in which the non-noise element is not adjusted.

### Claim 6 (depends on 1)

The image processing apparatus according to claim 1,
wherein the processor is configured to combine the first image and the second image at a ratio in which the excess and deficiency of the first AI processing is adjusted.

### Claim 7 (depends on 6)

The image processing apparatus according to claim 6,
wherein the processing target image is an image obtained by performing imaging by an imaging apparatus,
the first AI processing includes first correction processing of correcting a phenomenon, which appears in the processing target image due to a characteristic of the imaging apparatus, by using an AI method,
the first image includes a first corrected image obtained by performing the first correction processing, and
the processor is configured to adjust an element derived from the first correction processing by combining the first corrected image and the second image at the ratio.

### Claim 8 (depends on 7)

The image processing apparatus according to claim 7,
wherein the processor is configured to perform second correction processing of correcting the phenomenon by using a non-AI method,
the second image includes a second corrected image obtained by performing the second correction processing, and
the processor is configured to adjust the element derived from the first correction processing by combining the first corrected image and the second corrected image at the ratio.

### Claim 9 (depends on 7)

The image processing apparatus according to claim 7,
wherein the characteristic includes an optical characteristic of the imaging apparatus.

### Claim 10 (depends on 6)

The image processing apparatus according to claim 6,
wherein the first AI processing includes first change processing of changing a factor that controls a visual impression given from the processing target image by using an AI method,
the first image includes a first changed image obtained by performing the first change processing, and
the processor is configured to adjust an element derived from the first change processing by combining the first changed image and the second image at the ratio.

### Claim 11 (depends on 10)

The image processing apparatus according to claim 10,
wherein the processor is configured to perform second change processing of changing the factor by using a non-AI method,
the second image includes a second changed image obtained by performing the second change processing, and
the processor is configured to adjust the element derived from the first change processing by combining the first changed image and the second changed image at the ratio.

### Claim 12 (depends on 10)

The image processing apparatus according to claim 10,
wherein the factor includes a clarity, color, a gradation, a resolution, a blurriness, an emphasizing degree of an edge region, an image style, and/or an image quality related to skin.

### Claim 13 (depends on 6)

The image processing apparatus according to claim 6,
wherein the processing target image is a captured image obtained by imaging subject light, which is formed on a light-receiving surface by a lens of an imaging apparatus, by the imaging apparatus,
the first image includes a first aberration corrected image obtained by performing aberration region correction processing of correcting a region of the captured image where an aberration of the lens is reflected by using an AI method, as processing included in the first AI processing,
the second image includes a second aberration corrected image obtained by performing processing of correcting the region of the captured image where the aberration of the lens is reflected by using a non-AI method, and
the processor is configured to adjust an element derived from the aberration region correction processing by combining the first aberration corrected image and the second aberration corrected image at the ratio.

### Claim 14 (depends on 6)

The image processing apparatus according to claim 6,
wherein the first image includes a first colored image obtained by performing color processing of coloring a first region and a second region, which is a region different from the first region, with respect to the processing target image in a distinguishable manner by using an AI method, as processing included in the first AI processing,
the second image includes a second colored image obtained by performing processing of changing color of the processing target image by using a non-AI method, and
the processor is configured to adjust an element derived from the color processing by combining the first colored image and the second colored image at the ratio.

### Claim 15 (depends on 14)

The image processing apparatus according to claim 14,
wherein the second colored image is an image obtained by performing processing of coloring the first region and the second region with respect to the processing target image in a distinguishable manner by using the non-AI method.

### Claim 16 (depends on 14)

The image processing apparatus according to claim 14,
wherein the processing target image is an image obtained by imaging a first subject, and
the first region is a region where a specific subject, which is included in the first subject in the processing target image, is captured.

### Claim 17 (depends on 6)

The image processing apparatus according to claim 6,
wherein the first image includes a first contrast adjusted image obtained by performing first contrast adjustment processing of adjusting a contrast of the processing target image by using an AI method, as processing included in the first AI processing,
the second image includes a second contrast adjusted image obtained by performing second contrast adjustment processing of adjusting the contrast of the processing target image by using a non-AI method, and
the processor is configured to adjust an element derived from the first contrast adjustment processing by combining the first contrast adjusted image and the second contrast adjusted image at the ratio.

### Claim 18 (depends on 17)

The image processing apparatus according to claim 17,
wherein the processing target image is an image obtained by imaging a second subject,
the first contrast adjustment processing includes third contrast adjustment processing of adjusting the contrast of the processing target image according to the second subject by using the AI method,
the second contrast adjustment processing includes fourth contrast adjustment processing of adjusting the contrast of the processing target image according to the second subject by using the non-AI method,
the first image includes a third contrast image obtained by performing the third contrast adjustment processing,
the second image includes a fourth contrast image obtained by performing the fourth contrast adjustment processing, and
the processor is configured to adjust an element derived from the third contrast adjustment processing by combining the third contrast image and the fourth contrast image at the ratio.

### Claim 19 (depends on 17)

The image processing apparatus according to claim 17,
wherein the first contrast adjustment processing includes fifth contrast adjustment processing of adjusting a contrast between a center pixel included in the processing target image and a plurality of adjacent pixels adjacent to a vicinity of the center pixel by using the AI method,
the second contrast adjustment processing includes sixth contrast adjustment processing of adjusting the contrast between the center pixel and the plurality of adjacent pixels by using the non-AI method,
the first image includes a fifth contrast image obtained by performing the fifth contrast adjustment processing,
the second image includes a sixth contrast image obtained by performing the sixth contrast adjustment processing, and
the processor is configured to adjust an element derived from the fifth contrast adjustment processing by combining the fifth contrast image and the sixth contrast image at the ratio.

### Claim 20 (depends on 6)

The image processing apparatus according to claim 6,
wherein the first image includes a first resolution adjusted image obtained by performing first resolution adjustment processing of adjusting a resolution of the processing target image by using an AI method, as processing included in the first AI processing,
the second image includes a second resolution adjusted image obtained by performing second resolution adjustment processing of adjusting the resolution by using a non-AI method, and
the processor is configured to adjust an element derived from the first resolution adjustment processing by combining the first resolution adjusted image and the second resolution adjusted image at the ratio.

### Claim 21 (depends on 20)

The image processing apparatus according to claim 20,
wherein the first resolution adjustment processing is processing of performing a super-resolution on the processing target image by using the AI method, and
the second resolution adjustment processing is processing of performing the super-resolution on the processing target image by using the non-AI method.

### Claim 22 (depends on 6)

The image processing apparatus according to claim 6,
wherein the first image includes a first high dynamic range image obtained by performing expansion processing of expanding a dynamic range of the processing target image by using an AI method, as processing included in the first AI processing,
the second image includes a second high dynamic range image obtained by performing processing of expanding the dynamic range of the processing target image by using a non-AI method, and
the processor is configured to adjust an element derived from the expansion processing by combining the first high dynamic range image and the second high dynamic range image at the ratio.

### Claim 23 (depends on 6)

The image processing apparatus according to claim 6,
wherein the first image includes a first edge emphasized image obtained by performing emphasis processing of emphasizing an edge region in the processing target image more than a non-edge region, which is a region different from the edge region, by using an AI method, as processing included in the first AI processing,
the second image includes a second edge emphasized image obtained by performing processing of emphasizing the edge region more than the non-edge region by using a non-AI method, and
the processor is configured to adjust an element derived from the emphasis processing by combining the first edge emphasized image and the second edge emphasized image at the ratio.

### Claim 24 (depends on 6)

The image processing apparatus according to claim 6,
wherein the first image includes a first point image adjusted image obtained by performing point image adjustment processing of adjusting a blurriness amount of a point image with respect to the processing target image by using an AI method, as processing included in the first AI processing,
the second image includes a second point image adjusted image obtained by performing processing of adjusting the blurriness amount by using a non-AI method, and
the processor is configured to adjust an element derived from the point image adjustment processing by combining the first point image adjusted image and the second point image adjusted image at the ratio.

### Claim 25 (depends on 6)

The image processing apparatus according to claim 6,
wherein the processing target image is an image obtained by imaging a third subject,
the first image includes a first blurred image obtained by performing blur processing of applying a blurriness, which is determined in accordance with the third subject, to the processing target image by using an AI method, as processing included in the first AI processing,
the second image includes a second blurred image obtained by performing processing of applying the blurriness to the processing target image by using a non-AI method, and
the processor is configured to adjust an element derived from the blur processing by combining the first blurred image and the second blurred image at the ratio.

### Claim 26 (depends on 6)

The image processing apparatus according to claim 6,
wherein the first image includes a first round blurriness image obtained by performing round blurriness processing of applying a first round blurriness to the processing target image by using an AI method, as processing included in the first AI processing,
the second image includes a second round blurriness image obtained by performing processing of adjusting the first round blurriness from the processing target image by using a non-AI method or of applying a second round blurriness to the processing target image by using the non-AI method, and
the processor is configured to adjust an element derived from the round blurriness processing by combining the first round blurriness image and the second round blurriness image at the ratio.

### Claim 27 (depends on 6)

The image processing apparatus according to claim 6,
wherein the first image includes a first gradation adjusted image obtained by performing first gradation adjustment processing of adjusting a gradation of the processing target image by using an AI method, as processing included in the first AI processing,
the second image includes a second gradation adjusted image obtained by performing second gradation adjustment processing of adjusting the gradation of the processing target image by using a non-AI method, and
the processor is configured to adjust an element derived from the first gradation adjustment processing by combining the first gradation adjusted image and the second gradation adjusted image at the ratio.

### Claim 28 (depends on 27)

The image processing apparatus according to claim 27,
wherein the processing target image is an image obtained by imaging a fourth subject,
the first gradation adjustment processing is processing of adjusting the gradation of the processing target image according to the fourth subject by using the AI method, and
the second gradation adjustment processing is processing of adjusting the gradation of the processing target image according to the fourth subject by using the non-AI method.

### Claim 29 (depends on 6)

The image processing apparatus according to claim 6,
wherein the first image includes an image style changed image obtained by performing image style change processing of changing an image style of the processing target image by using an AI method, as processing included in the first AI processing, and
the processor is configured to adjust an element derived from the image style change processing by combining the image style changed image and the second image at the ratio.

### Claim 30 (depends on 6)

The image processing apparatus according to claim 6,
wherein the processing target image is an image obtained by imaging skin,
the first image includes a skin image quality adjusted image obtained by performing skin image quality adjustment processing of adjusting an image quality related to the skin captured in the processing target image by using an AI method, as processing included in the first AI processing, and
the processor is configured to adjust an element derived from the skin image quality adjustment processing by combining the skin image quality adjusted image and the second image at the ratio.

### Claim 31 (depends on 6)

The image processing apparatus according to claim 6,
wherein the first AI processing includes a plurality of purpose-specific processing performed by using an AI method,
the first image includes a multiple processed image obtained by performing the plurality of purpose-specific processing on the processing target image, and
the processor is configured to combine the multiple processed image and the second image at the ratio.

### Claim 32 (depends on 31)

The image processing apparatus according to claim 31,
wherein the plurality of purpose-specific processing are performed in an order based on a degree of influence applied on the processing target image.

### Claim 33 (depends on 32)

The image processing apparatus according to claim 32,
wherein the plurality of purpose-specific processing are performed stepwise from purpose-specific processing in which the degree of the influence is small to purpose-specific processing in which the degree of the influence is large.

### Claim 34 (depends on 6)

The image processing apparatus according to claim 6,
wherein the ratio is defined based on a difference between the processing target image and the first image and/or a difference between the first image and the second image.

### Claim 35 (depends on 6)

The image processing apparatus according to claim 6,
wherein the processor is configured to adjust the ratio according to related information that is related to the processing target image.

### Claim 36 (depends on 1)

An imaging apparatus comprising:
the image processing apparatus according to claim 1; and
an image sensor,
wherein the processing target image is an image obtained by performing imaging by the image sensor.

### Claim 37

An image processing method comprising:
acquiring a first image, which is obtained by performing first AI processing on a processing target image, and a second image, which is obtained without performing the first AI processing on the processing target image; and
adjusting excess and deficiency of the first AI processing by combining the first image and the second image.

### Claim 38

An image processing method comprising:
acquiring a first image, which is obtained by performing first AI processing on a processing target image to adjust a non-noise element of the processing target image, and a second image, which is obtained without performing the first AI processing on the processing target image; and
adjusting the non-noise element by combining the first image and the second image.

### Claim 39

A non-transitory computer-readable storage medium storing a program executable by a computer to perform a process comprising:
acquiring a first image, which is obtained by performing first AI processing on a processing target image, and a second image, which is obtained without performing the first AI processing on the processing target image; and
adjusting excess and deficiency of the first AI processing by combining the first image and the second image.

### Claim 40

A non-transitory computer-readable storage medium storing a program executable by a computer to perform a process comprising:
acquiring a first image, which is obtained by performing first AI processing on a processing target image to adjust a non-noise element of the processing target image, and a second image, which is obtained without performing the first AI processing on the processing target image; and
adjusting the non-noise element by combining the first image and the second image.
