# US 12,008,737 B2 · Deep learning model for noise reduction in low SNR imaging conditions > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,008,737 B2 - Document ID: US12008737B2 - Status: Granted - Kind: B2 - Application number: 17/444,499 - Filed: 2021-08-05 - Published: 2022-02-10 - Granted: 2024-06-11 - Earliest claimed: 2020-08-07 - Assignees: Nanotronics Imaging, Inc. - Applicants: Nanotronics Imaging, Inc. - Inventors: Denis Sharoukhov; Tonislav Ivanov; Jonathan Lee - Web page: https://patentgrep.com/patent/US12008737B2 ## CPC - G06T 5/70: Denoising; Smoothing - G06T 3/4046: using neural networks - G06T 5/50: using two or more images, e.g. averaging or subtraction - G06T 5/60: using machine learning, e.g. neural networks - G06T 2207/10016: Video; Image sequence - G06T 2207/10056: Microscopic image - G06T 2207/10064: Fluorescence image - G06T 2207/20081: Training; Learning - G06T 2207/20084: Artificial neural networks [ANN] - G06T 2207/30024: Cell structures in vitro; Tissue sections in vitro - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract Embodiments disclosed herein are generally related to a system for noise reduction in low signal to noise ratio imaging conditions. A computing system obtains a set of images of a specimen. The set of images includes at least two images of the specimen. The computing system inputs the set of images of the specimen into a trained denoising model. The trained denoising model is configured to output a single denoised image of the specimen. The computing system receives, as output from the trained denoising model, a single denoised image of the specimen. ## Claims ### Claim 1 A system, comprising: an imaging apparatus configured to perform darkfield imaging of a specimen; and a computing system in communication with the imaging apparatus, the computing system comprising one or more processors and a memory, the memory having programming coded thereon, which, when executed by the one or more processors, causes the computing system to perform operations comprising: obtaining, by the computing system, a plurality of noisy images of the specimen captured by the imaging apparatus using darkfield imaging, wherein the plurality of noisy images includes at least two images of the specimen captured using darkfield imaging; denoising, by the computing system, the plurality of noisy images by inputting the plurality of noisy images of the specimen into a convolutional neural network trained to output a single denoised image of the specimen; and generating, as output from the convolutional neural network, the single denoised image of the specimen. ### Claim 2 (depends on 1) The system of claim 1, wherein the operations further comprise: generating, by the computing system, a synthetic data set for training the convolutional neural network, the synthetic data set comprising a plurality of synthetic images and, for each synthetic image, a plurality of noisy images derived from the respective synthetic image; and training, by the computing system, the convolutional neural network to denoise a target plurality of noisy images based on the synthetic data set. ### Claim 3 (depends on 2) The system of claim 2, wherein the operations further comprise: generating, by the computing system, an empirical data set for finetuning the convolutional neural network following the training; and finetuning, by the computing system, the convolutional neural network based on the empirical data set. ### Claim 4 (depends on 3) The system of claim 3, wherein generating, by the computing system, the empirical data set for finetuning the convolutional neural network following the training comprises: generating a plurality of geometric objects to be placed inside a background image for capturing by the imaging apparatus; and receiving, from the imaging apparatus, a plurality of finetuning images based on the plurality of geometric objects placed inside the background image, the plurality of finetuning images defining the empirical data set. ### Claim 5 (depends on 4) The system of claim 4, wherein the operations further comprise: introducing shape irregularities to the plurality of geometric objects placed inside the background image. ### Claim 6 (depends on 1) The system of claim 1, wherein the convolutional neural network comprises a downsampling portion followed by an upsampling portion. ### Claim 7 (depends on 1) The system of claim 1, wherein the computing system is a component of the imaging apparatus. ### Claim 8 A method comprising: obtaining, by a computing system, a plurality of noisy images of a specimen captured by an imaging apparatus using darkfield imaging, wherein the plurality of noisy images includes at least two images of the specimen captured using darkfield imaging; denoising, by the computing system, the plurality of noisy images by inputting the plurality of noisy images of the specimen into a convolutional neural network trained to output a single denoised image of the specimen; and generating, as output from the convolutional neural network, the single denoised image of the specimen. ### Claim 9 (depends on 8) The method of claim 8, further comprising: generating, by the computing system, a synthetic data set for training the convolutional neural network, the synthetic data set comprising a plurality of synthetic images and, for each synthetic image, a plurality of noisy images derived from the respective synthetic image; and training, by the computing system, the convolutional neural network to denoise a target plurality of noisy images based on the synthetic data set. ### Claim 10 (depends on 9) The method of claim 9, further comprising: generating, by the computing system, an empirical data set for finetuning the convolutional neural network following the training; and finetuning, by the computing system, the convolutional neural network based on the empirical data set. ### Claim 11 (depends on 10) The method of claim 10, wherein generating, by the computing system, the empirical data set for finetuning the convolutional neural network following the training comprises: generating a plurality of geometric objects to be placed inside a background image for capturing by the imaging apparatus; and receiving, from the imaging apparatus, a plurality of finetuning images based on the plurality of geometric objects placed inside the background image, the plurality of finetuning images defining the empirical data set. ### Claim 12 (depends on 11) The method of claim 11, further comprising: introducing shape irregularities to the plurality of geometric objects placed inside the background image. ### Claim 13 (depends on 8) The method of claim 8, wherein the convolutional neural network comprises a downsampling portion followed by an upsampling portion. ### Claim 14 (depends on 8) The method of claim 8, wherein the computing system is a component of the imaging apparatus. ### Claim 15 A non-transitory computer readable medium comprising one or more sequences of instructions, which, when executed by one or more processors, causes a computing system to perform operations, comprising: obtaining, by the computing system, a plurality of noisy images of a specimen captured by an imaging apparatus using darkfield imaging, wherein the plurality of noisy images includes at least two images of the specimen captured using darkfield imaging; denoising, by the computing system, the plurality of noisy images by inputting the plurality of noisy images of the specimen into a convolutional neural network trained to output a single denoised image of the specimen; and generating, as output from the convolutional neural network, the single denoised image of the specimen. ### Claim 16 (depends on 15) The non-transitory computer readable medium of claim 15, further comprising: generating, by the computing system, a synthetic data set for training the convolutional neural network, the synthetic data set comprising a plurality of synthetic images and, for each synthetic image, a plurality of noisy images derived from the respective synthetic image; and training, by the computing system, the convolutional neural network to denoise a target plurality of noisy images based on the synthetic data set. ### Claim 17 (depends on 16) The non-transitory computer readable medium of claim 16, further comprising: generating, by the computing system, an empirical data set for finetuning the convolutional neural network following the training; and finetuning, by the computing system, the convolutional neural network based on the empirical data set. ### Claim 18 (depends on 17) The non-transitory computer readable medium of claim 17, wherein generating, by the computing system, the empirical data set for finetuning the convolutional neural network following the training comprises: generating a plurality of geometric objects to be placed inside a background image for capturing by the imaging apparatus; and receiving, from the imaging apparatus, a plurality of finetuning images based on the plurality of geometric objects placed inside the background image, the plurality of finetuning images defining the empirical data set. ### Claim 19 (depends on 18) The non-transitory computer readable medium of claim 18, further comprising: introducing shape irregularities to the plurality of geometric objects placed inside the background image. ### Claim 20 (depends on 15) The non-transitory computer readable medium of claim 15, wherein the convolutional neural network comprises a downsampling portion followed by an upsampling portion.