# US 2020/0402227 A1 · Image processing apparatus, and image processing method, and program

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0402227 A1
- Document ID: US20200402227A1
- Status: Published application
- Kind: A1
- Application number: 16/769,969
- Filed: 2018-11-30
- Published: 2020-12-24
- Earliest claimed: 2017-12-15
- Assignees: Sony Corporation
- Applicants: Sony Corporation
- Inventors: SHINICHIRO GOMI; ATSUSHI ITO; TAKESHI UEMORI
- Web page: https://patentgrep.com/patent/US20200402227A1

## CPC

- G06T 7/0012: {Biomedical image inspection}
- G16H 30/40: for processing medical images, e.g. editing
- A61B 5/00: Measuring for diagnostic purposes (radiation diagnosis A61B6/00; diagnosis by ultrasonic, sonic or infrasonic waves A61B8/00); Identification of persons
- G01B 11/303: {using photoelectric detection means}
- G01J 3/2823: {Imaging spectrometer}
- G06Q 50/22: Social work or social welfare, e.g. community support activities or counselling services
- G01J 2003/2873: {Storing reference spectrum}
- G06T 2207/10152: Varying illumination
- G06T 2207/30088: Skin; Dermal
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G01B: MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01J: MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY (light sources F21, H01J, H01K, H05B; investigating properties of materials by optical means G01N)
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR

## Abstract

The present disclosure relates to an image processing apparatus having a simple configuration and enabled to evaluate a state of a subject in view of an adverse effect of an ambient light source, and an image processing method, and a program. A difference square root sum between a subject spectral characteristic corresponding to a spectral characteristic of a skin of a user used as a subject in a captured image and a spectral characteristic of a preferred state is determined, and on the basis of comparison of the difference square root sum with a predetermined threshold, the state of the skin, used as a subject, is evaluated, and a recommended product corresponding to the evaluation result is presented. The present disclosure can be applied to a recommended product presenting apparatus.

## Claims

### Claim 1

An image processing apparatus comprising:
an evaluation section evaluating a state of a subject in a captured image on a basis of a subject spectral characteristic corresponding to a spectral characteristic of the subject and a reference spectral characteristic.

### Claim 2 (depends on 1)

The image processing apparatus according to claim 1, wherein
the evaluation section evaluates the state of the subject on a basis of a difference between the subject spectral characteristic and the reference spectral characteristic.

### Claim 3 (depends on 2)

The image processing apparatus according to claim 2, wherein
the evaluation section outputs, as an evaluation result for the state of the subject, a comparison result of comparison of a difference square root sum between the subject spectral characteristic and the reference spectral characteristic with a predetermined threshold.

### Claim 4 (depends on 3)

The image processing apparatus according to claim 3, wherein
the predetermined threshold is a difference square root sum between an average value of subject spectral characteristics of a plurality of subjects and the reference spectral characteristic.

### Claim 5 (depends on 1)

The image processing apparatus according to claim 1, wherein
the reference spectral characteristic is a spectral characteristic of a preferred state of the subject in the subject spectral characteristic.

### Claim 6 (depends on 1)

The image processing apparatus according to claim 1, further comprising:
an identification section identifying an article with a spectral characteristic applied to the subject to make the spectral characteristic of the subject more similar to the reference spectral characteristic, and
a display section displaying the article identified by the identification section.

### Claim 7 (depends on 6)

The image processing apparatus according to claim 6, wherein
in a case where, in an evaluation result for the state of the subject including a comparison result of comparison of a difference square root sum between the subject spectral characteristic and the reference spectral characteristic with the predetermined threshold, the difference square root sum between the subject spectral characteristic and the reference spectral characteristic is larger than the predetermined threshold,
the identification section identifies, on a basis of the difference between the subject spectral characteristic and the reference spectral characteristic, an article with a spectral characteristic applied to the subject to make the spectral characteristic of the subject more similar to the reference spectral characteristic.

### Claim 8 (depends on 6)

The image processing apparatus according to claim 6, wherein
the spectral characteristic applied to the subject to make the spectral characteristic of the subject more similar to the reference spectral characteristic is a spectral characteristic obtained by multiplying, for each wavelength, the difference between the subject spectral characteristic and the reference spectral characteristic by a predetermined coefficient and adding the subject spectral characteristic to a result of the multiplication.

### Claim 9 (depends on 6)

The image processing apparatus according to claim 6, further comprising:
an article storage section storing information related to the article in association with information related to a spectral characteristic provided in the subject by application of the article, wherein
the identification section identifies an article that is included in articles stored in the article storage section, the article involving a decrease in a difference square root sum between the spectral characteristic stored in association with the information and a spectral characteristic applied to the subject to make the spectral characteristic of the subject more similar to the reference spectral characteristic.

### Claim 10 (depends on 1)

The image processing apparatus according to claim 1, wherein
the evaluation section evaluates the state of the subject by classifying the subject spectral characteristic.

### Claim 11 (depends on 10)

The image processing apparatus according to claim 10, wherein
the evaluation section evaluates the state of the subject by dividing the subject spectral characteristic into a plurality of wavelength regions, classifying the subject spectral characteristic on a basis of a comparison result of comparison of a predetermined threshold with a difference square root sum for each wavelength region resulting from the division, and obtaining a classification result as an evaluation index.

### Claim 12 (depends on 11)

The image processing apparatus according to claim 11, further comprising:
an identification section identifying, on a basis of the evaluation index, an article with a spectral characteristic applied to the subject to make the spectral characteristic of the subject more similar to the reference spectral characteristic; and
a display section displaying the article identified by the identification section.

### Claim 13 (depends on 12)

The image processing apparatus according to claim 12, further comprising:
an article storage section storing, for information related to the article, an index of the article with a spectral characteristic applied to the subject to make the spectral characteristic of the subject more similar to the reference spectral characteristic, in association with an evaluation index of the subject spectral characteristic, the index of the article being based on a difference between the subject spectral characteristic and the reference spectral characteristic, wherein
the identification section identifies an article that is included in articles stored in the article storage section and that is stored in association with the evaluation index, as an article with a spectral characteristic applied to the subject to make the spectral characteristic of the subject more similar to the reference spectral characteristic.

### Claim 14 (depends on 13)

The image processing apparatus according to claim 13, wherein
the spectral characteristic applied to the subject to make the spectral characteristic of the subject more similar to the reference spectral characteristic is a spectral characteristic obtained by multiplying, for each wavelength, the difference between the subject spectral characteristic and the reference spectral characteristic by a predetermined coefficient and adding the subject spectral characteristic to a result of the multiplication.

### Claim 15 (depends on 1)

The image processing apparatus according to claim 1, further comprising:
an ambient-light-source spectral characteristic estimating section estimating a spectral characteristic of an ambient light source in a captured image as an ambient-light-source spectral characteristic, wherein
the evaluation section evaluates a state of the subject in an image with an adverse effect of the ambient light source in the image reduced using the ambient-light-source spectral characteristic estimated from the image, on a basis of a difference between the reference spectral characteristic and the subject spectral characteristic of the subject.

### Claim 16 (depends on 15)

The image processing apparatus according to claim 15, further comprising:
an ambient-light-source spectral characteristic storage section storing, in association with a measurement location, the ambient-light-source spectral characteristic estimated by the ambient-light-source spectral characteristic estimating section, wherein
the evaluation section evaluates, on a basis of the subject spectral characteristic of the subject, the state of the subject in an image with the adverse effect of the ambient light source in the image reduced using an ambient-light-source spectral characteristic selected from ambient-light-source spectral characteristics stored in the ambient-light-source spectral characteristic storage section.

### Claim 17 (depends on 15)

The image processing apparatus according to claim 15, further comprising:
an inappropriate-ambient-light-source detecting section detecting that the ambient light source is an inappropriate light source for the subject spectral characteristic on a basis of the ambient-light-source spectral characteristic estimated by the ambient-light-source spectral characteristic estimating section; and
a presentation section indicating that the ambient light source is an inappropriate light source in a case where the ambient light source is detected as an inappropriate light source.

### Claim 18 (depends on 17)

The image processing apparatus according to claim 17, wherein
the inappropriate-ambient-light-source detecting section detects that the light source is inappropriate on a basis of a comparison between an average value of the subject spectral characteristic in the ambient-light-source spectral characteristic estimated by the ambient-light-source spectral characteristic estimating section.

### Claim 19

An image processing method comprising:
evaluation processing of evaluating a state of a subject in a captured image on a basis of a reference spectral characteristic and a subject spectral characteristic corresponding to a spectral characteristic of the subject.

### Claim 20

A program causing an evaluation section evaluating a state of a subject in a captured image to function as a computer on a basis of a reference spectral characteristic and a subject spectral characteristic corresponding to a spectral characteristic of the subject.
