US 12,217,394 B2Granted

Method and apparatus for multicolor unmixing through iterative mutual-information minimization in simultaneous channel update structure

AssigneeKorea Advanced Institute of Science and Technology

InventorsJaeByum Chang, Junyoung Seo, Young-Gyu Yoon, Hyunwoo Kim, Jee Won Kim

Life of this patent

Estimating the term…

Loading patent…

Abstract

Methods and apparatuses for multicolor unmixing may involve iterative mutual information minimization in a simultaneous channel update structure. A plurality of images may be obtained with respect to a plurality of fluorophores labeled on different biomolecules, respectively. Images of the fluorophores may be unmixed from the obtained images, respectively, while simultaneously minimizing mutual information shared between the obtained images based on all permutation combinations of the obtained images. Signals having three or more fluorophores whose emission spectra are overlapped in one spectral range can be more accurately unmixed.

Claims16

Dependent claims

A method performed by an electronic device, comprising:

obtaining a plurality of images with respect to a plurality of fluorophores labeled on different biomolecules, respectively;
and
unmixing images of the fluorophores from the obtained images, respectively, while simultaneously minimizing mutual information shared between the obtained images based on all permutation combinations (NP2) of the obtained images;
wherein the unmixing of the images of the respective fluorophores comprises:
selecting two images of the obtained images based on one of all the permutation combinations (NP2);
updating an update function wherein updating of the update function comprises processing the selected images as low-resolution images having lower resolution than the selected images as in [Equation i] below;
and
u1=IMGi
u2=IMGj
v1=pixelBinning(u1,m)
v2=pixelBinning(u2,m) [Equation i]
wherein the u1 and the u2 indicate the obtained images, the v1 and the v2 indicate the low-resolution images, and the m indicates a factor applied in order to process the obtained images as the low-resolution images, respectively;
calculating the variable from the low-resolution images as in [Equation iii] by using a loss function defined as in [Equation ii] below;
and
L
⁡
(
α
)
=
I
⁡
(
v
⁢
⁢
1
;
v
⁢
⁢
2
-
α
×
v
⁢
⁢
1
)
[
Equation
⁢
⁢
ii
]
α
i
,
j
^
=
arg
⁢
⁢
min
α
⁢
⁢
L
⁡
(
α
)
=
arg
⁢
⁢
min
α
⁢
⁢
I
⁡
(
v
⁢
⁢
1
;
v
⁢
⁢
2
-
α
×
v
⁢
⁢
1
)
[
Equation
⁢
⁢
iii
]
wherein the indicates the variable;
updating the update function by applying the variable as in [Equation iv] below;
[
X
1
⁢
(
k
+
1
)
X
2
⁢
(
k
+
1
)
⋮
X
N
⁡
(
k
+
1
)
]
=
[
1
-
γ
⁢
α
^
1
,
2
⁢
(
k
)
…
-
γ
⁢
α
^
1
,
N
⁡
(
k
)
-
γ
⁢
α
^
2
,
1
⁢
(
k
)
1
⋱
⋮
⋮
⋱
⋱
-
γ
⁢
α
^
N
-
1
,
N
⁡
(
k
)
-
γ
⁢
α
^
N
,
1
⁢
(
k
)
…
-
γ
⁢
α
^
N
,
N
-
1
⁢
(
k
)
1
]
⁢
[
X
1
⁢
(
k
)
X
2
⁢
(
k
)
⋮
X
N
⁡
(
k
)
]
[
Equation
⁢
⁢
iv
]
wherein the k indicates an iteration number, the γ indicates an update ratio, and the Xi(k+1) indicates an i-th channel image after a k-th iteration;
and
obtaining a plurality of new images updated from the obtained images by using the update function, and Xi(k+1) obtained in a previous step is used as an input image IMGi.

244 more elements

An electronic device, comprising:

a memory;
and
a processor connected to the memory and configured to execute at least one instruction stored in the memory,
wherein the processor is configured to:
obtain a plurality of images with respect to a plurality of fluorophores labeled on different biomolecules, respectively;
and
unmix images of the fluorophores from the obtained images, respectively, while simultaneously minimizing mutual information shared between the obtained images based on all permutation combinations (NP2) of the obtained images;
wherein the processor is configured to:
select two images of the obtained images based on one of all the permutation combinations (NP2);
process the selected images as low resolution images having lower resolution than the selected images as in [Equation vi] below;
and
u1=IMGi
u2=IMGj
v1=pixelBinning(u1,m)
v2=pixelBinning(u2,m) [Equation vi]
wherein the u1 and the u2 indicate the obtained images, the v1 and the v2 indicate the low resolution images, and the m indicates a factor applied in order to process the obtained images as the low resolution images, respectively;
calculate the variable from the low resolution images as in [Equation viii] by using a loss function defined as in [Equation vii] below;
and
L
⁡
(
α
)
=
I
⁡
(
v
⁢
⁢
1
;
v
⁢
⁢
2
-
α
×
v
⁢
⁢
1
)
[
Equation
⁢
⁢
vii
]
=
arg
⁢
⁢
min
α
⁢
⁢
L
⁡
(
α
)
=
arg
⁢
⁢
min
α
⁢
⁢
I
⁡
(
v
⁢
⁢
1
;
v
⁢
⁢
2
-
α
×
v
⁢
⁢
1
)
[
Equation
⁢
⁢
viii
]
wherein the indicates the variable;
updating the update function by applying the variable as in [Equation ix] below;
[
X
1
⁢
(
k
+
1
)
X
2
⁢
(
k
+
1
)
⋮
X
N
⁡
(
k
+
1
)
]
=
[
1
-
γ
⁢
α
^
1
,
2
⁢
(
k
)
…
-
γ
⁢
α
^
1
,
N
⁡
(
k
)
-
γ
⁢
α
^
2
,
1
⁢
(
k
)
1
⋱
⋮
⋮
⋱
⋱
-
γ
⁢
α
^
N
-
1
,
N
⁡
(
k
)
-
γ
⁢
α
^
N
,
1
⁢
(
k
)
…
-
γ
⁢
α
^
N
,
N
-
1
⁢
(
k
)
1
]
⁢
[
X
1
⁢
(
k
)
X
2
⁢
(
k
)
⋮
X
N
⁡
(
k
)
]
[
Equation
⁢
⁢
ix
]
wherein the k indicates an iteration number, the γ indicates an update ratio, and the Xi(k+1) indicates an i-th channel image after a k-th iteration.

241 more elements

A non-transitory computer-readable storage medium for storing one or more programs for executing a method, the method comprising:

obtaining a plurality of images with respect to a plurality of fluorophores labeled on different biomolecules, respectively;
and
unmixing images of the fluorophores from the obtained images, respectively, while simultaneously minimizing mutual information shared between the obtained images based on all permutation combinations (NP2) of the obtained images;
wherein the unmixing of the images of the respective fluorophores comprises:
selecting two images of the obtained images based on one of all the permutation combinations (NP2);
updating an update function wherein updating of the update function comprises processing the selected images as low-resolution images having lower resolution than the selected images as in [Equation i] below;
and
u1=IMGi
u2=IMGj
v1=pixelBinning(u1,m)
v2=pixelBinning(u2,m) [Equation i]
wherein the u1 and the u2 indicate the obtained images, the v1 and the v2 indicate the low-resolution images, and the m indicates a factor applied in order to process the obtained images as the low-resolution images, respectively;
calculating the variable from the low-resolution images as in [Equation iii] by using a loss function defined as in [Equation ii] below;
and
L
⁡
(
α
)
=
I
⁡
(
v
⁢
⁢
1
;
v
⁢
⁢
2
-
α
×
v
⁢
⁢
1
)
[
Equation
⁢
⁢
ii
]
α
i
,
j
^
=
arg
⁢
⁢
min
α
⁢
⁢
L
⁡
(
α
)
=
arg
⁢
⁢
min
α
⁢
⁢
I
⁡
(
v
⁢
⁢
1
;
v
⁢
⁢
2
-
α
×
v
⁢
⁢
1
)
[
Equation
⁢
⁢
iii
]
wherein the indicates the variable;
updating the update function by applying the variable as in [Equation iv] below;
[
X
1
⁢
(
k
+
1
)
X
2
⁢
(
k
+
1
)
⋮
X
N
⁡
(
k
+
1
)
]
=
[
1
-
γ
⁢
α
^
1
,
2
⁢
(
k
)
…
-
γ
⁢
α
^
1
,
N
⁡
(
k
)
-
γ
⁢
α
^
2
,
1
⁢
(
k
)
1
⋱
⋮
⋮
⋱
⋱
-
γ
⁢
α
^
N
-
1
,
N
⁡
(
k
)
-
γ
⁢
α
^
N
,
1
⁢
(
k
)
…
-
γ
⁢
α
^
N
,
N
-
1
⁢
(
k
)
1
]
⁢
[
X
1
⁢
(
k
)
X
2
⁢
(
k
)
⋮
X
N
⁡
(
k
)
]
[
Equation
⁢
⁢
iv
]
wherein the k indicates an iteration number, the γ indicates an update ratio, and the Xi(k+1) indicates an i-th channel image after a k-th iteration;
and
obtaining a plurality of new images updated from the obtained images by using the update function, and Xi(k+1) obtained in a previous step is used as an input image IMGi.

244 more elements

Dependencies are inferred from the claim text.

Facts

Application: 17/567,300Filed 3 Jan 2022
Priority: KR 10-2021-00006885 Jan 2021
Published: US 2022/0215517 A17 Jul 2022
Granted: 4 Feb 2025
Term ends: Not estimatedThe source records don’t support an estimate.

Classification15 codes

G06T 5/50
Image enhancement or restoration: using two or more images, e.g. averaging or subtraction
G01N 21/6428
Fluorescence; Phosphorescence: measuring fluorescence of fluorescent products of reactions or of fluorochrome labelled reactive substances, e.g.…
G01N 21/6456
Specially adapted constructive features of fluorimeters: spatial resolved fluorescence measurements; Imaging
G06T 3/40
Geometric image transformations in the plane of the image: scaling of whole images or parts thereof, e.g. expanding or contracting

Family

US 2022/0215517 A1Pre-grant publication · published 7 Jul 2022
US 12,217,394 B2This patent

From the US filing history. Foreign family members aren’t listed.

Timeline

Citations

Open elsewhere

Building an agent or pulling many patents? Use the API or MCP server instead of scraping pages. Get an API key