# US 2019/0320126 A1 · Image processing method, image processing device, and storage medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0320126 A1
- Document ID: US20190320126A1
- Status: Published application
- Kind: A1
- Application number: 16/468,743
- Filed: 2017-12-15
- Published: 2019-10-17
- Earliest claimed: 2016-12-22
- Assignees: NEC Corporation
- Applicants: NEC Corporation
- Inventors: Ryuichi AKASHI
- Web page: https://patentgrep.com/patent/US20190320126A1

## CPC

- G02B 5/208: {for use with infrared or ultraviolet radiation, e.g. for separating visible light from infrared and/or ultraviolet radiation}
- H04N 23/11: for generating image signals from visible and infrared light wavelengths
- H04N 23/12: with one sensor only
- H04N 23/13: with multiple sensors
- H04N 23/60: Control of cameras or camera modules
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

An image processing device (10) is provided with: an estimation unit (11) for estimating, by using image data and a spectral sensitivity characteristic of a color image sensor in a wavelength range from visible light to near-infrared light, a spectral distribution characteristic of an incident light incident on the color image sensor, based on a model expressing the spectral distribution characteristic of the incident light, the image data including one or more color channels and acquired by capturing, using the color image sensor, the incident light including visible light and near-infrared light; and a generation unit (12) for generating, using the estimated spectral distribution characteristic of the incident light and the spectral sensitivity characteristic of the color image sensor, visible light image data configured only from information representing the visible light and near-infrared light image data configured only from information representing the near-infrared light.

## Claims

### Claim 1

An image processing method comprising:
estimating, by using image data and a spectral sensitivity characteristic of a color image sensor in a wavelength range from visible light to near-infrared light, a spectral distribution characteristic of an incident light incident on the color image sensor, based on a model expressing the spectral distribution characteristic of the incident light, the image data including one or more color channels and acquired by capturing, using the color image sensor, the incident light including visible light and near-infrared light; and
generating, by using the estimated spectral distribution characteristic of the incident light and the spectral sensitivity characteristic of the color image sensor, visible light image data composed only of information representing visible light and near-infrared light image data composed only of information representing near-infrared light, wherein
each integral value of the spectral sensitivity characteristic of the color image sensor in a wavelength range of near-infrared light relating to each of the one or more color channels of the color image sensor satisfies a predetermined condition.

### Claim 2 (depends on 1)

The image processing method according to claim 1, further comprising:
generating the visible light image data composed only of information representing visible light, by using a product of the estimated spectral distribution characteristic of the incident light by the spectral sensitivity characteristic of the color image sensor in a wavelength range of visible light; and
generating the near-infrared light image data composed only of information representing near-infrared light, by using a product of the spectral distribution characteristic of the incident light by the spectral sensitivity characteristic of the color image sensor in a wavelength range of near-infrared light.

### Claim 3 (depends on 1)

The image processing method according to claim 1, further comprising:
generating the visible light image data composed only of information representing visible light, by using a product of the estimated spectral distribution characteristic of the incident light by the spectral sensitivity characteristic of the color image sensor in a wavelength range of visible light; and
generating the near-infrared light image data composed only of information representing near-infrared light, by subtracting the generated visible light image data from the image data in which visible light and near-infrared light are captured.

### Claim 4 (depends on 1)

The image processing method according to claim 1, further comprising:
generating the near-infrared light image data composed only of information representing near-infrared light, by using a product of the estimated spectral distribution characteristic of the incident light by the spectral sensitivity characteristic of the color image sensor in a wavelength range of near-infrared light; and
generating the visible light image data composed only of information representing visible light, by subtracting the generated near-infrared light image data from the image data in which visible light and near-infrared light are captured.

### Claim 5 (depends on 1)

The image processing method according to claim 1, wherein
spectral distribution characteristic of incident light is modeled by a weighted sum of basis vectors ri(λ) (i=1 to M, where M is an integer of 1 or greater), is modeled by a weighted sum of basis vectors ri(λ) (i=1 to M+1), is modeled by the weighted sum of the basis vectors ri(λ) (i=1 to M) and a mean vector r0(λ) of the spectral distribution characteristic, or is modeled by the weighted sum of the basis vectors ri(λ) (i=1 to M+1) and the mean vector r0(λ).

### Claim 6 (depends on 1)

The image processing method according to claim 1, wherein
the one or more color channels included in the image data are a red channel, a green channel, and a blue channel.

### Claim 7

An image processing device comprising:
at least one memory storing a program; and
at least one processor configured to execute the program to perform:
estimating, by using image data and a spectral sensitivity characteristic of a color image sensor in a wavelength range from visible light to near-infrared light, a spectral distribution characteristic of an incident light incident on the color image sensor, based on a model expressing the spectral distribution characteristic of the incident light, the image data including one or more color channels and acquired by capturing, using the color image sensor, the incident light including visible light and near-infrared light; and
generating, by using the estimated spectral distribution characteristic of the incident light and the spectral sensitivity characteristic of the color image sensor, visible light image data composed only of information representing visible light and near-infrared light image data composed only of information representing near-infrared light, wherein
each integral value of the spectral sensitivity characteristic of the color image sensor in a wavelength range of near-infrared light relating to each of the one or more color channels of the color image sensor satisfies a predetermined condition.

### Claim 8 (depends on 7)

The image processing device according to claim 7, wherein
the at least one processor is configured to perform:
generating the visible light image data composed only of information representing visible light, by using a product of the estimated spectral distribution characteristic of the incident light by the spectral sensitivity characteristic of the color image sensor in a wavelength range of visible light, and
generating the near-infrared light image data composed only of information representing near-infrared light, by using a product of the spectral distribution characteristic of the incident light by the spectral sensitivity characteristic of the color image sensor in a wavelength range of near-infrared light.

### Claim 9

A non-transitory storage medium recording an image processing program for causing a computer to execute:
estimation processing of estimating, by using image data and a spectral sensitivity characteristic of a color image sensor in a wavelength range from visible light to near-infrared light, a spectral distribution characteristic of an incident light incident on the color image sensor, based on a model expressing the spectral distribution characteristic of the incident light, the image data including one or more color channels and acquired by capturing, using the color image sensor, the incident light including visible light and near-infrared light; and
generation processing of generating, by using the estimated spectral distribution characteristic of the incident light and the spectral sensitivity characteristic of the color image sensor, visible light image data composed only of information representing visible light and near-infrared light image data composed only of information representing near-infrared light, wherein
each integral value of the spectral sensitivity characteristic of the color image sensor in a wavelength range of near-infrared light relating to each of the one or more color channels of the color image sensor satisfies a predetermined condition.

### Claim 10 (depends on 9)

The storage medium according to claim 9, wherein
the image processing program causes a computer to execute:
first generation processing of generating the visible light image data composed only of information representing visible light, by using a product of the estimated spectral distribution characteristic of the incident light by the spectral sensitivity characteristic of the color image sensor in a wavelength range of visible light; and
second generation processing of generating the near-infrared light image data composed only of information representing near-infrared light, by using a product of the estimated spectral distribution characteristic of the incident light by the spectral sensitivity characteristic of the color image sensor in a wavelength range of near-infrared light.
