# US 12,593,985 B2 · Image processing apparatus, image processing method, and non-transitory computer-readable medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,593,985 B2
- Document ID: US12593985B2
- Status: Granted
- Kind: B2
- Application number: 17/178,403
- Filed: 2021-02-18
- Published: 2021-06-17
- Granted: 2026-04-07
- Earliest claimed: 2018-08-21
- Assignees: Canon Kabushiki Kaisha; Luxonus Inc.
- Applicants: Canon Kabushiki Kaisha; Keio University
- Inventors: Shoya Sasaki; Kenichi Nagae; Hiroki Kajita; Nobuaki Imanishi; Sadakazu Aiso
- Web page: https://patentgrep.com/patent/US12593985B2

## CPC

- A61B 5/0095: {by applying light and detecting acoustic waves, i.e. photoacoustic measurements}
- A61B 5/14542: {for measuring blood gases (A61B5/14551 takes precedence)}
- A61B 5/0075: {by spectroscopy, i.e. measuring spectra, e.g. Raman spectroscopy, infrared absorption spectroscopy (A61B5/0071 takes precedence)}
- A61B 5/742: {using visual displays (displays for heart-related electrical signals, e.g. ECG, A61B5/339)}
- A61B 5/0091: {for mammography}
- A61B 8/466: {adapted to display 3D data}
- A61B 8/481: {involving the use of contrast agents, e.g. microbubbles introduced into the bloodstream}
- A61B 8/483: {involving the acquisition of a 3D volume of data}
- A61B 2576/00: Medical imaging apparatus involving image processing or analysis (A61B1/00009, A61B6/52 and A61B8/52 take precedence)
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)

## Abstract

An image processing apparatus includes a spectral image acquisition unit configured to acquire a spectral image that is an image generated using photoacoustic signals corresponding to a plurality of different wavelengths, based on photoacoustic waves generated by radiating lights with the plurality of different wavelengths to an object into which a contrast agent has been injected, a contrast agent information acquisition unit configured to acquire information about the contrast agent, a region determination unit configured to determine a region corresponding to the contrast agent in the spectral image on the basis of the information about the contrast agent, and a display control unit configured to display the spectral image such that the region corresponding to the contrast agent is distinguishable from a region other than the region corresponding to the contrast agent.

## Claims

### Claim 1

An image processing apparatus comprising:
a memory storing a program; and
one or more processors which, by executing the program, function as a plurality of units comprising:
(1) a spectral image signal acquisition unit configured to acquire a single spectral image signal by performing a common calculation processing on both of (a) a first photoacoustic signal obtained by irradiating an object with light of a first wavelength and (b) a second photoacoustic signal obtained by irradiating the object with light of a second wavelength different from the first wavelength, wherein the common calculation processing includes a common formula using both of the first photoacoustic signal and the second photoacoustic signal, and wherein as a result of the common calculation processing, the single spectral image signal has (a) a first numerical value range corresponding to the contrast agent, (b) a second numerical value range corresponding to oxyhemoglobin, and (c) a third numerical value range corresponding to deoxyhemoglobin;
(2) a region determination unit configured to (a) determine a region of the single spectral image signal having the first numerical value range in the single spectral image signal as a lymphatic vessel region, (b) determine a region of the single spectral image signal having the second numerical value range in the single spectral image signal as an artery region, and (c) determine a region of the single spectral image signal having the third numeral value range in the single spectral image signal as a vein region; and
(3) a display control unit configured to display the single spectral image signal in a manner based on image values of the single spectral image signal included in the first numerical value range, the second numerical value range, and the third numerical value range such that the lymphatic vessel region, the artery region, and the vein region are distinguishable from each other,
wherein the first wavelength and the second wavelength are set such that the first numerical value range, the second numerical value range, and the third numerical value range are different from one another.

### Claim 2 (depends on 1)

The image processing apparatus according to claim 1, wherein the region determination unit is configured to:
determine the first numerical value range on the basis of information about the contrast agent, and
determine a region of the single spectral image having image values included in the first numerical value range as a region corresponding to the contrast agent.

### Claim 3 (depends on 1)

The image processing apparatus according to claim 1, wherein the region determination unit is configured to determine the first numerical value range for image values of the single spectral image to which colors are allocated on the basis of information about the contrast agent, and
wherein the display control unit is configured to allocate colors to image values of the single spectral image signal included in the first numerical value range and to display the single spectral image.

### Claim 4 (depends on 1)

The image processing apparatus according to claim 1, wherein the image processing apparatus is configured such that a sign of an image value in the first numerical value range and a sign of an image value in the second numerical value range are different.

### Claim 5 (depends on 1)

The image processing apparatus according to claim 1, wherein the region determination unit is configured to determine the region corresponding to the contrast agent in the single spectral image signal on the basis of information about the contrast agent and information on the plurality of wavelengths.

### Claim 6

An image processing apparatus comprising:
a memory storing a program; and
one or more processors which, by executing the program, function as a plurality of units comprising:
(1) a spectral image signal acquisition unit configured to a acquire a single spectral image signal by performing a common calculation processing on both of (a) a first photoacoustic signal obtained by irradiating an object with light of a first wavelength and (b) a second photoacoustic signal obtained by irradiating the object with light of a second wavelength different from the first wavelength, wherein the common calculation processing includes a common formula using both of the first photoacoustic signal and the second photoacoustic signal, and wherein as a result of the common calculation processing, the single spectral image signal has (a) a first numerical value range corresponding to the contrast agent, (b) a second numerical value range corresponding to oxyhemoglobin, and (c) a third numerical value range corresponding to deoxyhemoglobin;
(2) a region determination unit configured to (a) determine a region of the single spectral image signal having the first numerical value range in the single spectral image signal as a lymphatic vessel region, (b) determine a region of the single spectral image signal having the second numerical value range in the single spectral image signal as an artery region, and (c) determine a region of the single spectral image signal having the third numeral value range in the single spectral image signal as a vein region; and
(3) a display control unit configured to allocate different colors to image values of the single spectral image signal included in the first numerical value range, the second numerical value range, and the third numerical value range and to display the single spectral image signal in which the lymphatic vessel region, the artery region, and the vein region are displayed in the different colors to be distinguishable from each other,
wherein the first wavelength and the second wavelength are set such that the first numerical value range, the second numerical value range, and the third numerical value range are different from one another.

### Claim 7 (depends on 6)

The image processing apparatus according to claim 6, wherein the display control unit is configured to allocate colors to image values of the single spectral image signal and displays the single spectral image signal on the basis of information about the contrast agent and information on the plurality of wavelengths.

### Claim 8 (depends on 1)

The image processing apparatus according to claim 1, wherein the display control unit is configured to:
display a color scale indicating a relation between image values and display colors of the single spectral image signal, and
display the color scale such that a display color corresponding to the contrast agent is able to be identified.

### Claim 9 (depends on 1)

The image processing apparatus according to claim 1, wherein the spectral image signal acquisition unit is configured to acquire the single spectral image signal by radiating lights with the plurality of wavelengths to the object into which the contrast agent has been injected, and
wherein the display control unit is configured to:
(1) determine a brightness of the single spectral image signal on the basis of image values of the single spectral image signal, and
(2) determine a conversion table for conversion from image values of the single spectral image signal into brightness on the basis of image values of the single spectral image signal.

### Claim 10 (depends on 1)

The image processing apparatus according to claim 1, wherein the spectral image signal acquisition unit is configured to (1) acquire a first photoacoustic image based on photoacoustic waves generated according to radiation of light with a first wavelength and a second photoacoustic image based on photoacoustic waves generated according to radiation of light with a second wavelength, and (2) generate an image signal based on a ratio of the first photoacoustic image to the second photoacoustic image as the single spectral image signal.

### Claim 11 (depends on 1)

The image processing apparatus according to claim 1, wherein, in a case where a measured value based on photoacoustic waves generated according to radiation of light with a first wavelength λ1 of the plurality of different wavelengths is Iλ1(r), a measured value based on photoacoustic waves generated according to radiation of light with a second wavelength λ2 of the plurality of different wavelengths is Iλ2(r), a molar absorption coefficient of deoxyhemoglobin corresponding to the first wavelength λ1 is εHbλ1, absorption coefficient of deoxyhemoglobin corresponding to the second wavelength λ2 is εHbλ2, a molar absorption coefficient of oxyhemoglobin corresponding to the first wavelength λ1 is εHbOλ1, a molar absorption coefficient of oxyhemoglobin corresponding to the second wavelength λ2 is εHbOλ2, and r is a position, the spectral image signal acquisition unit is configured to generate a calculated value Is(r) of the single spectral image signal according to the following formula:
Is
⁡
(
r
)
=
I
λ
2
⁡
(
r
)
I
λ
1
⁡
(
r
)
⁣
·
ε
H
⁢
⁢
b
λ
1
-
ε
H
⁢
⁢
b
λ
2
(
ε
H
⁢
⁢
bO
λ
2
-
ε
H
⁢
⁢
b
λ
2
)
-
I
λ
2
⁡
(
r
)
I
λ
1
⁡
(
r
)
·
(
ε
H
⁢
⁢
bO
λ
1
-
ε
H
⁢
⁢
b
λ
1
)
[
Math
.
⁢
1
]

### Claim 12 (depends on 11)

The image processing apparatus according to claim 11, wherein the measured values are absorption coefficients or initial sound pressures.

### Claim 13 (depends on 1)

The image processing apparatus according to claim 1, wherein the contrast agent is indocyanine green (ICG) having a concentration of at least 2.5 mg/mL and not more than 10.0 mg/mL.

### Claim 14 (depends on 1)

The image processing apparatus according to claim 1, wherein information representing a concentration of the contrast agent is acquired as information about the contrast agent on the basis of an instruction from a user, and
wherein in a case where information representing that a type of the contrast agent is indocyanine green (ICG) is acquired as the information about the contrast agent, an instruction from the user which indicates an ICG concentration less than 2.5 mg/mL or greater than 10.0 mg/mL is not received.

### Claim 15 (depends on 13)

The image processing apparatus according to claim 13, wherein the plurality of wavelengths include 797 nm as a first wavelength and 835 nm as a second wavelength.

### Claim 16 (depends on 1)

The image processing apparatus according to claim 1, wherein the display control unit is configured to display a plurality of spectral images generated in a time series as a moving image.

### Claim 17 (depends on 16)

The image processing apparatus according to claim 16, wherein the display control unit is able to display the moving image in a fast-forward manner.

### Claim 18 (depends on 16)

The image processing apparatus according to claim 16, wherein the display control unit is able to repeatedly display the moving image.

### Claim 19

An image processing method comprising:
acquiring a single spectral image signal by performing a common calculation processing on both of (a) a first photoacoustic signal obtained by irradiating an object with light of a first wavelength and (b) a second photoacoustic signal obtained by irradiating the object with light of a second wavelength different from the first wavelength, wherein the common calculation processing includes a common formula using both of the first photoacoustic signal and the second photoacoustic signal, and wherein as a result of the common calculation processing, the single spectral image signal has (a) a first numerical value range corresponding to the contrast agent, (b) a second numerical value range corresponding to oxyhemoglobin, and (c) a third numerical value range corresponding to deoxyhemoglobin;
determining a region of the single spectral image signal having the first numerical value range in the single spectral image signal as a lymphatic vessel region;
determining a region of the single spectral image signal having the second numerical value range in the single spectral image signal as an artery region;
determining a region of the single spectral image signal having the third numerical value range in the single spectral image signal as a vein region; and
displaying the single spectral image signal in a manner based on image values of the single spectral image signal included in the first numerical value range, the second numerical value range, and the third numerical value range such that the lymphatic vessel region, the artery region, and the vein region are distinguishable from each other,
wherein the acquiring comprises setting the first wavelength and the second wavelength such that the first numerical value range, the second numerical value range, and the third numerical value range are different from one another.

### Claim 20

An image processing method comprising:
acquiring a single spectral image signal by performing a common calculation processing on both of (a) a first photoacoustic signal obtained by irradiating an object with light of a first wavelength and (b) a second photoacoustic signal obtained by irradiating the object with light of a second wavelength different from the first wavelength, wherein the common calculation processing includes a common formula using both of the first photoacoustic signal and the second photoacoustic signal, and wherein as a result of the common calculation processing, the single spectral image signal has (a) a first numerical value range corresponding to the contrast agent, (b) a second numerical value range corresponding to oxyhemoglobin, and (c) a third numerical value range corresponding to deoxyhemoglobin;
determining a region of the single spectral image signal having the first numerical value range in the single spectral image signal as a lymphatic vessel region;
determining a region of the single spectral image signal having the second numerical value range in the single spectral image signal as an artery region;
determining a region of the single spectral image signal having the third numerical value range in the single spectral image signal as a vein region;
allocating different colors to image values of the single spectral image signal included in the first numerical value range, the second numerical value range, and the third numerical value range; and
displaying the single spectral image signal in which the lymphatic vessel region, the artery region, and the vein region are displayed in the different colors to be distinguishable from each other,
wherein the acquiring comprises setting the first wavelength and the second wavelength such that the first numerical value range, the second numerical value range, and the third numerical value range are different from one another.

### Claim 21 (depends on 19)

The image processing method according to claim 19, wherein the contrast agent is indocyanine green (ICG) and the plurality of wavelengths include 797 nm as a first wavelength and 835 nm as a second wavelength.

### Claim 22 (depends on 19)

A non-transitory computer-readable medium storing a program for causing a computer to execute the image processing method according to claim 19.

### Claim 23 (depends on 20)

A non-transitory computer-readable medium storing a program for causing a computer to execute the image processing method according to claim 20.

### Claim 24 (depends on 1)

The image processing apparatus according to claim 1, wherein at least one of the first wavelength and the second wavelength is a wavelength at which the molar absorption coefficient of deoxyhemoglobin becomes greater than the molar absorption coefficient of oxyhemoglobin for imaging of a vein.

### Claim 25 (depends on 1)

The image processing apparatus according to claim 1, wherein each of the first wavelength and the second wavelength is a wavelength at which the molar absorption coefficient of deoxyhemoglobin becomes greater than the molar absorption coefficient of oxyhemoglobin for imaging of a vein.

### Claim 26 (depends on 1)

The image processing apparatus according to claim 1, wherein at least one of the first wavelength and the second wavelength is a wavelength at which the absorption coefficient of the contrast agent is less than the absorption coefficient of the blood.

### Claim 27 (depends on 1)

The information processing apparatus according to claim 1, wherein a calculated value Is(r) of the single spectral image signal is determined as a function of a ratio of (a) a measured value based on photoacoustic waves generated according to radiation of light with the second wavelength and (b) a measured value based on photoacoustic waves generated according to radiation of light with the first wavelength.

### Claim 28 (depends on 27)

The information processing apparatus according to claim 27, wherein the calculated value Is(r) is determined as a function of (1) the ratio of (a) a measured value based on photoacoustic waves generated according to radiation of light with the second wavelength and (b) a measured value based on photoacoustic waves generated according to radiation of light with the first wavelength and (2) a value determined by taking the negative of that ratio.

### Claim 29 (depends on 28)

The information processing apparatus according to claim 28, wherein the first wavelength and the second wavelength are both wavelengths at which a molar absorption coefficient of deoxyhemoglobin is greater than a molar absorption coefficient of oxyhemoglobin, and
wherein at least one of the first wavelength and the second wavelength is a wavelength at which an absorption coefficient of the contrast agent is less than an absorption coefficient of blood.

### Claim 30 (depends on 28)

The information processing apparatus according to claim 28, wherein the first numerical value range corresponding to the contrast agent includes both (a) values that are smaller than those within the second numerical value range corresponding to oxyhemoglobin and (b) values that are larger than those within the second numerical value range corresponding to oxyhemoglobin and the third numerical value range corresponding to deoxyhemoglobin.

### Claim 31 (depends on 28)

The information processing apparatus according to claim 28, wherein the first numerical value range corresponding to the contrast agent has a sign opposite to that of the second numerical value range corresponding to oxyhemoglobin and the third numerical value range corresponding to deoxyhemoglobin.