US 12,521,010 B2 · Portable system for identifying potential cases of diabetic macular oedema using image processing and artificial intelligence

Abstract

Diabetes is a disease characterized by high levels of blood glucose. It is important to keep diabetes under control to avoid short- and long-term complications. Diabetes can affect vision due to the alterations it produces in the blood vessels of the retina. This is known as Diabetic Retinopathy (DR), which is one of the leading causes of impaired vision in developed countries. One of the complications of diabetic retinopathy is Diabetic Macular Edema (DME), which is the leading cause of vision loss in diabetic patients and can appear at any stage of diabetic retinopathy. This consists of the gradual accumulation of fluid in the macula, the most important area of the retina. The determination of diabetic macular oedema is very important for the retina. The determination of diabetic macular oedema is very important for adequate treatment of this condition. A variety of technological options are used for detecting diabetic retinopathy, although only the most sophisticated detect macular oedema, a complication that appears as a consequence of diabetic retinopathy and is one of the leading causes of blindness. The invention describes a portable system for detecting diabetic macular oedema by capturing a fundus image using a portable ophthalmoscope; said image is sent via wired or wireless means to an embedded system that has an algorithm based on artificial intelligence, which extracts information from the image and processes same to identify the presence of the condition being studied.

A portable system for detecting potential cases of diabetic macular oedema by processing a fundus image comprising:

a. a portable ophthalmoscopy device to capture the fundus image comprising:
i. a continuous field angle of 45° to 30°, +35 D [[ . . . 11 to −35 D,
ii. a 40 mm lens and at least 4 infrared light emitting diodes (IR LEDs) each having a maximum power of 100 mW,
iii. a thin-film-transistor liquid-crystal display (TFT LCD) touch screen approximately 3.97 inches and uses approximately 800×480 pixels and 16.7 million colors further having an anti-reflective film designed to display an internal image of a human eye;
iv. a universal serial bus (USB) connection and/or a short-range wireless connection used for exchanging data between mobile devices over a short distance,
v. a capture button, wherein said capture button retains an image for processing and analysis, and
vi. a power button,
b. a portable, embedded imaging processing and analysis device having
a development card;
a liquid-crystal display (LCD) screen;
and
a universal serial bus (USB) or micro universal serial bus (USB) port and/or a short-range wireless connection module, wherein the development card is designed to process an image received from the portable ophthalmoscopic device using the serial bus USB connections of both the portable ophthalmoscopic device and the portable, embedded imaging processing and analysis device and/or the short-range wireless connection that is a short-range wireless interconnection of both the portable ophthalmoscopic device and the portable, embedded imaging processing and analysis device,
wherein the liquid-crystal display (LCD) screen of the portable, embedded image processing and analysis device displays the processes image received from the portable ophthalmoscopic device;
and
wherein said imaging processing and analysis device with the development card further comprises:
i. an electrical connector,
ii. a light emitting diode (LED) power indicator,
iii. a power button, and
iv. a microprocessor designed to process the information contained in the image received from the portable ophthalmoscopic device, wherein the microprocessor is configured to read the image transmitted by the portable ophthalmoscopic device to create a greyscale conversion of the image received from the portable ophthalmoscopic device, and
c. an artificial intelligence (AI)-based algorithm embedded in the portable, embedded imaging processing and analysis device that uses artificial intelligence for the in situ processing and analysis of the image of a diabetic macular oedema received from the portable ophthalmoscopy device that is in direct communication with the portable ophthalmoscopy device, wherein said algorithm comprises the following steps:
i. reading of the image by wired or wireless means,
ii. converting the image to grayscale,
iii. automatically selecting a region through detection of characteristics corresponding to vascular intersections and thereafter cropping the image;
iv. automatically detecting the vascular intersections using a search algorithm based on changes in intensity gradients in the image components;
v. applying an error minimization process by performing the following steps:
1. Decomposing, scaling, and skeletonizing the image,
2. Searching the image for changes in gradient intensity through gradient changes and morphologically searching for branches,
3. Filling a vector of characteristics,
4. Minimizing errors,
5. Counting intersection points,
vi. binarizing the image,
vii. determining whether or not noise has been eliminated,
viii. removing noise in the image using morphological operations and automatically identifying objects in the image having an area that is smaller than 200 pixels to create a noise-free image,
ix. skeletonizing the noise-free image using morphological operations, identifying vascular intersections and measuring the distance between the identified vascular intersections,
x. rigid body transforming consisting of image scaling, rotating and translating,
xi. image feature segmenting and extracting,
xii. applying the artificial intelligence system:
1. extracting statistical characteristics selected from the group consisting of energy, entropy, homogeneity, contrast, brightness, correlation, average RGB, average R, average G, and average B of the image,
2. Classifying by using a neural network, comprising the steps of:
1) Estimating the thickness of a macula;
and
2) Classifying the estimated thickness of the macula;
3. Pre-diagnosing a diabetic macular oedema;
wherein the system detects potential cases of diabetic macular oedema as the system is designed to perform the following functions:
a. capturing an image of a fundus of an eye, wherein the image has a resolution of at least 5 megapixels (MP) and sized 1536×1152 pixels and the image is formatted as a Joint Photographic Experts Group (JPEG) or Tag Image File Format (TIFF) image file format,
b. sending the fundus image to an embedded system by wired or wireless means or storing the fundus image in a universal serial bus (USB) and/or micro-universal serial bus (USB) memory flash drive, wherein the USB memory flash drive is afterwards inserted into a port of the portable, embedded imaging processing and analysis device,
c. achieving an embedded system through image processing and extracting characteristics of the embedded system for classification, and
d. displaying pathology information on a liquid crystal display (LCD) screen;
wherein the greyscale conversion by the microprocessor of the fundus image received from-the portable ophthalmoscopic device, comprises the steps of
considering a digital RGB fundus image represented by Ieye(m;
n), where Ieye is a M×N image for m=1, 2, . . . , M and n=1, 2, . . . N;
decomposing the fundus image into color components Rm,n, Gm,n Bm,n;
wherein each component, Pm,n is a specific pixel with a value from 0 to L, wherein L represents the maximum intensity value, and
automatically selecting a region from the captured image by detecting features corresponding to vascular intersections from which the image is cropped;
automatically detecting vascular intersections by performing a search process based on the changes of the intensity gradients in the image components;
processing of error minimization, comprising the steps of:
decomposing, scaling and skeletonizing image components;
searching for intensity changes by means of gradient changes and morphological branch search;
filling of the characteristics vector;
minimizing errors;
counting of intersection points and definition of Ieye;
defining Ieye,
applying a cropping operation on Ieye to generate an image Iroi(i;
j), wherein Iroi⊂Ieye for
l
gs
(
i
,
j
)
=
∑
s
=
1
3
⁢
∑
t
=
1
3
⁢
K
s
(
s
,
t
)
⁢
l
rg
(
i
+
s
,
j
+
t
)
∑
s
=
-
1
3
⁢
∑
t
=
-
1
3
⁢
K
s
(
s
·
t
)
α≤i≤M and β≤j≤N, α≥1, β≥1, α, β∈, and further wherein, Iroi must contain two intersections of eye vessels, p1(xp1, yp1) and p2(xp2, yp2) as follows:
applying a median filter using the following steps:
converting Iori(i, j) to Img(i, j), a greyscale representation where px,y∈[0, L] for data set reduction;
applying a mean kernel (Ks) on Irg to reduce noise, wherein Igs is the averaged image and k22 is the kernel,
performing a binarization of the image using the following steps:
K
=
[
k
11
k
12
k
13
k
21
k
22
k
23
k
31
k
32
k
33
]
⁢
K
s
=
1
9
[
1
1
1
1
1
1
1
1
1
]
applying during image processing, a gradient using directional kernels, wherein KGx and KGy are directional kernels of dimension 3×3, wherein said gradient is
I
gm
=
(
K
Gx
(
I
gs
)
)
2
+
(
K
Gy
(
I
gs
)
)
2
obtaining the magnitude of the image gradient, where Igm identifies the local illumination variation and the direction of maximum gray intensity changes for Igs;
determining if the noise has been removed from the image using the following steps:
binarizing the Igs images using Otsu's method to generate a black and white image Ibw;
obtaining Ifn which is the resulting noise-free image;
performing a noise removal on the image through morphological operations and automatic identification of objects called noise (objects with are a less than 200 pixels);
obtaining Ifn which is the resulting image free of noise;
skeletonizing the noise-free image Ifn through morphological operations,
identifying vascular intersections and distance measurement between them, comprising the following steps:
skeletonizing Ifn using mathematical morphological operations;
removing pixels at the boundaries of the objects,
maintaining the objects together;
preserving the Euler number;
storing spatial coordinates of the intersections;
obtaining Isk as a final image;
performing a rigid body transformation consisting of scaling, rotating and translating the image;
performing image segmentation and feature extraction;
performing statistical feature extraction to obtain characteristics of the image thereby obtaining energy, entropy, homogeneity, contrast, brightness, correlation, RGB mean, R-mean, G-mean and B-mean;
and
using an artificial neural network classifier to estimate the thickness of the macula and classifying a result.

207 more elements

Portable system for identifying potential cases of diabetic macular oedema using image processing and artificial intelligence

Life of this patent

Portable system for identifying potential cases of diabetic macular oedema using image processing and artificial intelligence

Life of this patent

Abstract

Claims4