Analyzing Complex Single Molecule Emission Patterns with Deep Learning
AssigneePurdue Research Foundation
InventorsPeiyi ZHANG, Fang HUANG, Sheng LIU
Life of this patent
Estimating the term…
Abstract
A fluorescent single molecule emitter simultaneously transmits its identity, location, and cellular context through its emission patterns. A deep neural network (DNN) performs multiplexed single-molecule analysis to enable retrieving such information with high accuracy. The DNN can extract three-dimensional molecule location, orientation, and wavefront distortion with precision approaching the theoretical limit of information content of the image which will allow multiplexed measurements through the emission patterns of a single molecule.
Claims20
A computer-implemented method for training a neural network for use in extracting physical information from an image of point spread function (PSF) emission patterns of illuminated single molecules, the emission patterns corresponding to ground truth parameters θtn indicative of the physical information of the molecule, the method comprising:
- (a) acquiring a dataset of a plurality of N×N pixel images An of the PSF emission patterns and providing the dataset to a processor;
- (b) forward propagating the dataset through a neural network, with the processor, comprising a plurality of convolutional and residual layers l and a plurality of training parameters wl in the layers l, to generate an output vector of a number of physical parameters {circumflex over (θ)}tn in a kth iteration through the neural network;
- (c) determining with the processor a measure of the information content of the dataset regarding the number of physical parameters;
- (d) comparing, with the processor, the output physical parameters {circumflex over (θ)}tn to the ground truth parameters θtn by calculating a mean squared error (MSE) weighted by the measure of information content based on the equation
- E
- θ
- ^
- =
- 1
- NT
-
- ∑
- n
- =
- 1
- N
-
- ∑
- t
- =
- 1
- T
-
- (
- θ
- ^
- tn
- -
- θ
- tn
- )
- 2
- Measure
- θ
- tn
- ,
- where N is the number of images An, T is the size of the output vector for each image, and Measureθtn is the value for parameter θt of image An of the information content of the dataset;
- (e) updating, with the processor, the parameters wl of the layers l of the neural network based on the derivative of the CRLB-weighted MSE according to the equation
- w
- k
- +
- 1
- l
- =
- w
- k
- l
- -
- η
- M
-
- ∑
- n
- =
- 1
- M
-
- ∂
- E
- θ
- ^
-
- n
- ,
- k
- ∂
- w
- n
- ,
- k
- l
- ,
- where l=0,1,2, . . . L denotes the layers in the neural network, η is the learning rate, k is the iteration number, and M is the number of images An;
- (f) repeating steps (c)-(e) for successive iterations until the error E{circumflex over (θ)} reaches a predetermined limit, after which the parameters wl of the layers l of the neural network are assigned, with the processor, the updated values of the last iteration for subsequently using the neural network, with the processor, to identify unknown physical parameters from the PSF of a new image of a single molecule.
A computer-implemented method for training a neural network for use in extracting physical information from point spread function (PSF) emission patterns of illuminated single molecules, the emission patterns corresponding to ground truth parameters θtn indicative of the physical information of the molecule, the method comprising:
- (a) acquiring a dataset of a plurality of N×N pixel images An of the PSF emission patterns and providing the dataset to a processor;
- (b) forward propagating the dataset through a neural network, with the processor, comprising a plurality of convolutional and residual layers l and a plurality of training parameters wl in the layers l, to generate an output vector of a number of physical parameters {circumflex over (θ)}m in a kth iteration through the neural network;
- (c) comparing, with the processor, the output physical parameters {circumflex over (θ)}tn to the ground truth parameters θtn by calculating a Cramér-Rao lower bound (CRLB)-weighted mean squared error (MSE) based on the equation
- E
- θ
- ^
- =
- 1
- NT
-
- ∑
- n
- =
- 1
- N
-
- ∑
- t
- =
- 1
- T
-
- (
- θ
- ^
- tn
- -
- θ
- tn
- )
- 2
- CRLB
- θ
- tn
- ,
- where N is the number of images An, T is the size of the output vector for each image, and CRLBθtn is the value of the Cramér-Rao lower bound (CRLB) for parameter θt of image An;
- (d) updating, with the processor, the parameters wl of the layers l of the neural network based on the derivative of the CRLB-weighted MSE according to the equation
- w
- k
- +
- 1
- l
- =
- w
- k
- l
- -
- η
- M
-
- ∑
- n
- =
- 1
- M
-
- ∂
- E
- θ
- ^
-
- n
- ,
- k
- ∂
- w
- n
- ,
- k
- l
- ,
- where l=0,1,2, . . . L denotes the layers in the neural network, η is the learning rate, k is the iteration number, and M is the number of images An;
- (e) repeating steps (b)-(d) until the error E{circumflex over (θ)} reaches a predetermined limit, after which the parameters wl of the layers l of the neural network are assigned, with the processor, the updated values of the last iteration for subsequently using the neural network, with the processor, to identify unknown physical parameters from the PSF of new image of a single molecule.
A computer-implemented method for identifying unknown physical parameters from the point spread function (PSF) of an image of a single molecule, the physical parameters including one or more of axial (z) position, lateral (x, y) position, wavefront distortion and dipole orientation of the single molecule, the method comprising:
- (a) training a neural network for use in extracting physical information from PSF emission patterns of illuminated single molecules, the emission patterns corresponding to ground truth parameters θtn indicative of the known physical information of the molecule, by;
- (i) acquiring a dataset of a plurality of N×N pixel images An of the PSF emission patterns corresponding to ground truth parameters θtn and providing the dataset to a processor;
- (ii) forward propagating the dataset through a neural network, with the processor, comprising a plurality of convolutional and residual layers l and a plurality of training parameters wl in the layers l, to generate an output vector of a number of physical parameters {circumflex over (θ)}tn in a kth iteration through the neural network;
- (iii) comparing, with the processor, the output physical parameters {circumflex over (θ)}tn to the ground truth parameters θtn and calculating a Cramér-Rao lower bound (CRLB)-weighted mean squared error (MSE) based on the equation
- E
- θ
- ^
- =
- 1
- NT
-
- ∑
- n
- =
- 1
- N
-
- ∑
- t
- =
- 1
- T
-
- (
- θ
- ^
- tn
- -
- θ
- tn
- )
- 2
- CRLB
- θ
- tn
- ,
- where N is the number of images An, T is the size of the output vector for each image, and CRLBθtn is the value of the Cramér-Rao lower bound (CRLB) for parameter θt of image An;
- (iv) updating, with the processor, the parameters wl of the layers l of the neural network based on the derivative of the CRLB-weighted MSE according to the equation
- w
- k
- +
- 1
- l
- =
- w
- k
- l
- -
- η
- M
-
- ∑
- n
- =
- 1
- M
-
- ∂
- E
- θ
- ^
-
- n
- ,
- k
- ∂
- w
- n
- ,
- k
- l
- ,
- where l=0,1,2, . . . L denotes the layers in the neural network, η is the learning rate, k is the iteration number, and M is the number of images An;
- (v) repeating steps (ii)-(iv) until the error E{circumflex over (θ)} reaches a predetermined limit, after which the parameters wl of the layers l of the neural network are assigned, with the processor, the updated values of the last iteration;
- and
- (b) acquiring a new dataset of an N×N pixel images of the PSF emission patterns corresponding to the unknown physical parameters and providing the dataset to the processor;
- and
- (c) forward propagating the new dataset through the trained neural network to generate an output vector with values corresponding to the one or more physical parameters.
Dependencies are inferred from the claim text.