US 11,379,958 B2Granted

Diagnosis assisting device, and image processing method in diagnosis assisting device

AssigneeCasio Computer Co., Ltd.

InventorKazuhisa Matsunaga

Life of this patent

Estimating the term…

Abstract

Provided are a diagnosis assisting device that facilitates a user to grasp a difference of an affected area to perform a highly precise diagnosis assistance, an image processing method in the diagnosis assisting device, and a program. An image processing method in a diagnosis assisting device that diagnoses lesions from a picked-up image includes (A) performing an image processing on the picked-up image. In (A), a peripheral area other than a diagnosis area that has a high probability as diseases in the picked-up image is set to be a measuring area when an image correction is performed.

Claims14

Dependent claims

An image processing method in a diagnosis assisting device that diagnoses tissue lesions from a picked-up image, the method comprising:

setting a center circle of the picked-up image as a diagnosis area;
setting a peripheral tissue area other than the diagnosis area in the picked-up image to be a measuring area to measure information relating to at least one of a color and a brightness in the peripheral tissue area;
measuring the information relating to at least one of the color and the brightness in the peripheral tissue area;
and
performing an image correction on the picked-up image, wherein performing the image correction comprises performing at least a color correction,
wherein performing the color correction comprises:
obtaining, based on the measured information in the measuring area, for each of R, G, and B pixels within the measuring area other than the center circle, an Lp norm of the corresponding pixel, according to arithmetic expression (1)-(3), where the Lp norms obtained for each of the R, G, and B pixels are normalized by a coefficient k such that L2 norms of eR, eG and eB become 1 as expressed by arithmetic expression (3), and where IC represents an original pixel value of a given one of the R, G, and B pixels;
calculating color correction gains dR, dG and dB for R, G, and B pixels, respectively, according to arithmetic expression (4);
performing a gain balance adjustment by adjusting color correction gains dR, dG and dB according to arithmetic expressions (6), (7), and (8) to obtain gain coefficients dR2, dG2 and dB2,
multiplying original pixel values IR, IG, IB of each of the R, G, B pixel values by the corresponding gain coefficients dR2, dG2 and dB2 in a form represented by arithmetic expression (5) to obtain corrected pixel values ItR, ItG, ItB, to thereby perform the color correction as the image correction on the picked-up image and generate a corrected image, and
outputting the corrected image,
wherein arithmetic expressions (1)-(8) are defined as:
(
∫
(
I
c
⁡
(
x
)
)
p
⁢
d
⁢
⁢
x
∫
d
⁢
⁢
x
)
1
/
p
=
k
⁢
⁢
e
c
(
1
)
c
∈
{
R
.
⁢
G
.
⁢
B
}
(
2
)
e
R
2
+
e
G
2
+
e
B
2
=
1
(
3
)
d
c
=
1
3
⁢
e
c
(
4
)
(
I
R
t
I
G
t
I
B
t
)
=
(
d
R
0
0
0
d
G
0
0
0
d
B
)
⁢
(
I
R
I
G
I
B
)
(
5
)
d
R
⁢
2
=
d
R
*
1.0
(
6
)
d
G
⁢
2
=
d
G
*
0.9
(
7
)
d
B
⁢
2
=
d
B
*
1.0
.
(
8
)

159 more elements

An image processing method in a diagnosis assisting device that diagnoses lesions from a picked-up image, the method comprising:

performing an image normalization on the picked-up image for diagnosis to normalize pixel values of the picked-up image based on a normalization target value that is set based on a human skin color;
and
after having performed the image normalization on the picked-up image, subtracting the normalization target value used in the image normalization from the normalized pixel values of the picked-up image having undergone the image normalization, to thereby obtain an input image to an identifier that identifies diseases based on image data relating to an object to be diagnosed,
wherein performing the image normalization comprises performing at least a color correction,
wherein performing the color correction comprises:
obtaining, based on measured information relating to color in a measuring area of the picked-up image, for each of R, G, and B pixels within the measuring area, an Lp norm of the corresponding pixel, according to arithmetic expression (1)-(3), where the Lp norms obtained for each of the R, G, and B pixels are normalized by a coefficient k such that L2 norms of eR, eG and eB become 1 as expressed by arithmetic expression (3), and where IC represents an original pixel value of a given one of the R, G, and B pixels;
calculating color correction gains dR, dG and dB for R, G, and B pixels, respectively, according to arithmetic expression (4);
performing a gain balance adjustment by adjusting color correction gains dR, dG and dB according to arithmetic expressions (6), (7), and (8) to obtain gain coefficients dR2, dG2 and dB2,
multiplying original pixel values IR, IG, IB of each of the R, G, B pixel values by the corresponding gain coefficients dR2, dG2 and dB2 in a form represented by arithmetic expression (5) to obtain corrected pixel values ItR, ItG, ItB to thereby perform the color correction as the image normalization on the picked-up image,
wherein arithmetic expressions (1)-(8) are defined as:
(
∫
(
I
c
⁡
(
x
)
)
p
⁢
d
⁢
⁢
x
∫
d
⁢
⁢
x
)
1
/
p
=
k
⁢
⁢
e
c
(
1
)
c
∈
{
R
.
⁢
G
.
⁢
B
}
(
2
)
e
R
2
+
e
G
2
+
e
B
2
=
1
(
3
)
d
c
=
1
3
⁢
e
c
(
4
)
(
I
R
t
I
G
t
I
B
t
)
=
(
d
R
0
0
0
d
G
0
0
0
d
B
)
⁢
(
I
R
I
G
I
B
)
(
5
)
d
R
⁢
2
=
d
R
*
1.0
(
6
)
d
G
⁢
2
=
d
G
*
0.9
(
7
)
d
B
⁢
2
=
d
B
*
1.0
.
(
8
)

157 more elements

An image processing method in a diagnosis assisting device that diagnoses lesions from a picked-up image, the method comprising:

performing an image normalization on the picked-up image for diagnosis to normalize pixel values of the picked-up image based on a normalization target value that is set based on a human skin color;
and
subtracting an average pixel value of the picked-up image from the normalized pixel values of the picked-up image having already undergone the image normalization based on the normalization target value, the normalization target value being different from the average pixel value of the picked-up image, to obtain an input image to an identifier that identifies diseases based on image data relating to an object to be diagnosed, the average pixel value being obtained by globally averaging the pixel values of all the pixels of the picked-up image,
wherein performing the image normalization comprises performing at least a color correction, and
wherein performing the color correction comprises:
obtaining, based on measured information relating to color in a measuring area of the picked-up image, for each of R, G, and B pixels within the measuring area, an Lp norm of the corresponding pixel, according to arithmetic expression (1)-(3), where the Lp norms obtained for each of the R, G, and B pixels are normalized by a coefficient k such that L2 norms of eR, eG and eB become 1 as expressed by arithmetic expression (3), and where IC represents an original pixel value of a given one of the R, G, and B pixels;
calculating color correction gains dR, dG and dB for R, G, and B pixels, respectively, according to arithmetic expression (4);
performing a gain balance adjustment by adjusting color correction gains dR, dG and dB according to arithmetic expressions (6), (7), and (8) to obtain gain coefficients dR2, dG2 and dB2,
multiplying original pixel values IR, IG, IB of each of the R, G, B pixel values by the corresponding gain coefficients dR2, dG2 and dB2 in a form represented by arithmetic expression (5) to obtain corrected pixel values ItR, ItG, ItB to thereby perform the color correction as the image normalization on the picked-up image,
wherein arithmetic expressions (1)-(8) are defined as:
(
∫
(
I
c
⁡
(
x
)
)
p
⁢
d
⁢
⁢
x
∫
d
⁢
⁢
x
)
1
/
p
=
k
⁢
⁢
e
c
(
1
)
c
∈
{
R
.
⁢
G
.
⁢
B
}
(
2
)
e
R
2
+
e
G
2
+
e
B
2
=
1
(
3
)
d
c
=
1
3
⁢
e
c
(
4
)
(
I
R
t
I
G
t
I
B
t
)
=
(
d
R
0
0
0
d
G
0
0
0
d
B
)
⁢
(
I
R
I
G
I
B
)
(
5
)
d
R
⁢
2
=
d
R
*
1.0
(
6
)
d
G
⁢
2
=
d
G
*
0.9
(
7
)
d
B
⁢
2
=
d
B
*
1.0
.
(
8
)

157 more elements

A diagnosis assisting device that diagnoses tissue lesions from a picked-up image, the device comprising:

a processor that is configured to:
set a center circle of the picked-up image as a diagnosis area;
set a peripheral tissue part other than the diagnosis area in the picked-up image as a measuring area to measure information relating to at least one of a color and a brightness in the peripheral tissue area;
and
perform an image correction on the picked-up image for diagnosis, wherein the processor is configured to selectively perform a color correction and a brightness correction as the image correction;
wherein, in performing the color correction as the image correction, the processor is configured to:
obtain, based on the measured information in the measuring area, for each of R, G, and B pixels within the measuring area other than the center circle, an Lp norm of the corresponding pixel, according to arithmetic expression (1)-(3), where the Lp norms obtained for each of the R, G, and B pixels are normalized by a coefficient k such that L2 norms of eR, eG and eB become 1 as expressed by arithmetic expression (3), and where IC represents an original pixel value of a given one of the R, G, and B pixels;
calculate color correction gains dR, dG and dB for R, G, and B pixels, respectively, according to arithmetic expression (4);
perform a gain balance adjustment by adjusting color correction gains dR, dG and dB according to arithmetic expressions (6), (7), and (8) to obtain gain coefficients dR2, dG2 and dB2;
multiply original pixel values IR, IG, IB of each of the R, G, B pixel values by the corresponding gain coefficients dR2, dG2 and dB2 in a form represented by arithmetic expression (5) to obtain corrected pixel values ItR, ItG, ItB, to thereby perform the color correction as the image correction on the picked-up image and generate a corrected image;
and
output the corrected image, wherein arithmetic expressions (1)-(8) are defined as:
(
∫
(
I
c
⁡
(
x
)
)
p
⁢
d
⁢
⁢
x
∫
d
⁢
⁢
x
)
1
/
p
=
k
⁢
⁢
e
c
(
1
)
c
∈
{
R
.
⁢
G
.
⁢
B
}
(
2
)
e
R
2
+
e
G
2
+
e
B
2
=
1
(
3
)
d
c
=
1
3
⁢
e
c
(
4
)
(
I
R
t
I
G
t
I
B
t
)
=
(
d
R
0
0
0
d
G
0
0
0
d
B
)
⁢
(
I
R
I
G
I
B
)
(
5
)
d
R
⁢
2
=
d
R
*
1.0
(
6
)
d
G
⁢
2
=
d
G
*
0.9
(
7
)
d
B
⁢
2
=
d
B
*
1.0
.
(
8
)

159 more elements

Dependencies are inferred from the claim text.

Facts

Application: 15/642,280Filed 5 Jul 2017
Priority: JP JP2016-1715952 Sep 2016
Published: US 2018/0068426 A18 Mar 2018
Granted: 5 Jul 2022
Term ends: Not estimatedThe source records don’t support an estimate.

Classification21 codes

G06T 7/0012
Inspection of images, e.g. flaw detection: biomedical image inspection
A61B 5/444
Skin evaluation, e.g. for skin disorder diagnosis: evaluating skin marks, e.g. mole, nevi, tumour, scar
A61B 5/7264
Details of waveform analysis: classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems…
G06T 5/40
Image enhancement or restoration: using histogram techniques

Family

US 2018/0068426 A1Pre-grant publication · published 8 Mar 2018
US 11,379,958 B2This patent

From the US filing history. Foreign family members aren’t listed.

Timeline

Citations

Open elsewhere

patentgrep

Loading patent…

Abstract

An image processing method in a diagnosis assisting device that diagnoses tissue lesions from a picked-up image, the method comprising:

setting a center circle of the picked-up image as a diagnosis area;
setting a peripheral tissue area other than the diagnosis area in the picked-up image to be a measuring area to measure information relating to at least one of a color and a brightness in the peripheral tissue area;
measuring the information relating to at least one of the color and the brightness in the peripheral tissue area;
and
performing an image correction on the picked-up image, wherein performing the image correction comprises performing at least a color correction,
wherein performing the color correction comprises:
obtaining, based on the measured information in the measuring area, for each of R, G, and B pixels within the measuring area other than the center circle, an Lp norm of the corresponding pixel, according to arithmetic expression (1)-(3), where the Lp norms obtained for each of the R, G, and B pixels are normalized by a coefficient k such that L2 norms of eR, eG and eB become 1 as expressed by arithmetic expression (3), and where IC represents an original pixel value of a given one of the R, G, and B pixels;
calculating color correction gains dR, dG and dB for R, G, and B pixels, respectively, according to arithmetic expression (4);
performing a gain balance adjustment by adjusting color correction gains dR, dG and dB according to arithmetic expressions (6), (7), and (8) to obtain gain coefficients dR2, dG2 and dB2,
multiplying original pixel values IR, IG, IB of each of the R, G, B pixel values by the corresponding gain coefficients dR2, dG2 and dB2 in a form represented by arithmetic expression (5) to obtain corrected pixel values ItR, ItG, ItB, to thereby perform the color correction as the image correction on the picked-up image and generate a corrected image, and
outputting the corrected image,
wherein arithmetic expressions (1)-(8) are defined as:
(
∫
(
I
c
⁡
(
x
)
)
p
⁢
d
⁢
⁢
x
∫
d
⁢
⁢
x
)
1
/
p
=
k
⁢
⁢
e
c
(
1
)
c
∈
{
R
.
⁢
G
.
⁢
B
}
(
2
)
e
R
2
+
e
G
2
+
e
B
2
=
1
(
3
)
d
c
=
1
3
⁢
e
c
(
4
)
(
I
R
t
I
G
t
I
B
t
)
=
(
d
R
0
0
0
d
G
0
0
0
d
B
)
⁢
(
I
R
I
G
I
B
)
(
5
)
d
R
⁢
2
=
d
R
*
1.0
(
6
)
d
G
⁢
2
=
d
G
*
0.9
(
7
)
d
B
⁢
2
=
d
B
*
1.0
.
(
8
)

159 more elements

Facts

Application

15/642,280Filed 5 Jul 2017

Priority

JP JP2016-1715952 Sep 2016

Published

US 2018/0068426 A18 Mar 2018

Granted

5 Jul 2022

Term ends

Not estimatedThe source records don’t support an estimate.

Classification21 codes

G06T 7/0012
Inspection of images, e.g. flaw detection: biomedical image inspection
A61B 5/444
Skin evaluation, e.g. for skin disorder diagnosis: evaluating skin marks, e.g. mole, nevi, tumour, scar
A61B 5/7264
Details of waveform analysis: classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems…
G06T 5/40
Image enhancement or restoration: using histogram techniques