# US 10,319,079 B2 · Noise estimation using bracketed image capture > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,319,079 B2 - Document ID: US10319079B2 - Status: Granted - Kind: B2 - Application number: 15/639,531 - Filed: 2017-06-30 - Published: 2019-01-03 - Granted: 2019-06-11 - Earliest claimed: 2017-06-30 - Assignees: Microsoft Technology Licensing, LLC - Applicants: Microsoft Technology Licensing, LLC - Inventors: Sudhanshu Sohoni - Web page: https://patentgrep.com/patent/US10319079B2 ## CPC - G06T 5/70: Denoising; Smoothing - G06T 5/50: using two or more images, e.g. averaging or subtraction - H04N 23/743: Bracketing, i.e. taking a series of images with varying exposure conditions - H04N 23/76: by influencing the image signals - H04N 23/81: for suppressing or minimising disturbance in the image signal generation - G06T 2207/20024: Filtering details - G06T 2207/20182: Noise reduction or smoothing in the temporal domain; Spatio-temporal filtering - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract Methods and devices for reducing noise in an image may include receiving an in focus image and at least one out of focus image of a scene, wherein the out of focus image has a second value of at least one focus-related parameter offset from a first value of the at least one focus-related parameter used for the in focus image. The methods and devices may include estimating a noise model of noise in at least a section of the out of focus image, and determining at least one parameter value for a filter based on the noise model. The methods and devices may include generating a noise-reduced image by applying the filter with the at least one parameter value to the in focus image to reduce at least part of the noise from at least a corresponding section of the in focus image. ## Claims ### Claim 1 A computer device, comprising: a memory to store data and instructions; a processor in communication with the memory; an operating system in communication with the memory and the processor, wherein the operating system is operable to: receive an in focus image and at least one out of focus image of a scene, wherein the out of focus image has a second value of at least one focus-related parameter offset from a first value of the at least one focus-related parameter used for the in focus image; estimate a noise model of noise in at least a section of the out of focus image based at least in part on identifying a frequency spread of one or more frequency components and amplitudes of the noise; determine at least one parameter value for a filter based on the noise model; and generate a noise-reduced image by applying the filter with the at least one parameter value to the in focus image to reduce at least part of the noise in at least a corresponding section of the in focus image. ### Claim 2 (depends on 1) The computer device of claim 1, wherein the operating system is further operable to estimate the noise model by using a frequency analysis to determine a pixel variance in at least the section of the out of focus image. ### Claim 3 (depends on 2) The computer device of claim 2, wherein the frequency analysis includes a frequency transform applied to the out of focus image to identify frequency components and corresponding amplitudes of the noise in at least the section of the out of focus image. ### Claim 4 (depends on 1) The computer device of claim 1, wherein to generate the noise-reduced image, the operating system is further operable to: determine one or more regions of interest of the out of focus image; estimate a respective noise model of the noise for each of the one or more regions of interest; determine one or more respective parameter values for the filter for each of the one or more regions of interest based on the respective noise model for each of the region of interests; and apply the filter with the one or more respective parameter values to a corresponding region of interest in the in focus image. ### Claim 5 (depends on 1) The computer device of claim 1, wherein to estimate the noise model of the noise in the out of focus image, the operating system is further operable to: divide the out of focus image into a plurality of sections; apply a frequency transform to each of the plurality of sections to identify the one or more frequency components and the amplitudes of the noise in the respective section; and identify the frequency spread of the one or more frequency components and the amplitudes of the noise in the respective section. ### Claim 6 (depends on 5) The computer device of claim 5, wherein to generate the noise-reduced image, the operating system is further operable to: identify for removal at least a portion of the frequency spread of the one or more frequency components and amplitudes of the noise in each of the plurality of sections; determine a respective one or more parameter values for the filter corresponding to at least the portion of the frequency spread of the one or more frequency components and amplitudes of the noise in each of the plurality of sections; and apply the filter with the respective one or more parameter values to at least a corresponding one of each of a plurality of sections of the in focus image. ### Claim 7 (depends on 1) The computer device of claim 1, wherein the filter with the at least one parameter value applies an inverse of at least a portion of noise components of the noise identified by the noise model. ### Claim 8 (depends on 1) The computer device of claim 1, wherein the in focus image and the out of focus image are captured in low light. ### Claim 9 (depends on 1) The computer device of claim 1, wherein the operating system is further operable to: estimate a second noise model of noise in at least a section of the in focus image corresponding to at least the section of the out of focus image, wherein the noise model of the out of focus image and the second noise model of the in focus image include frequency components; identify a common set of frequency components in both the noise model of the out of focus image and the second noise model of the in focus image; wherein to determine the at least one parameter value for the filter the operating system is further operable to determine one or more parameter values based on the common set of frequency components; and wherein to generate the noise-reduced image, the operating system is further operable to apply the filter with the one or more parameter values to the in focus image. ### Claim 10 (depends on 1) The computer device of claim 1, wherein an amount of noise removed from portions of the in focus image varies based on a measured noise for each portion of the in focus image. ### Claim 11 A method for reducing noise in an image, comprising: receiving, at an operating system executing on a computer device, an in focus image and at least one out of focus image of a scene, wherein the out of focus image has a second value of at least one focus-related parameter offset from a first value of the at least one focus-related parameter used for the in focus image; estimating, by the operating system, a noise model of noise in at least a section of the out of focus image based at least in part on identifying a frequency spread of one or more frequency components and amplitudes of the noise; determining at least one parameter value for a filter based on the noise model; and generating a noise-reduced image by applying the filter with the at least one parameter value to the in focus image to reduce at least part of the noise in at least a corresponding section of the in focus image. ### Claim 12 (depends on 11) The method of claim 11, wherein estimating the noise model further comprises using a frequency analysis to determine a pixel variance in at least the section of the out of focus image. ### Claim 13 (depends on 12) The method of claim 12, wherein the frequency analysis includes a frequency transform applied to the out of focus image to identify frequency components and corresponding amplitudes of the noise in at least the section of the out of focus image. ### Claim 14 (depends on 11) The method of claim 11, wherein estimating the amount of noise further comprises: determining one or more regions of interest of the out of focus image; estimating a respective noise model of the noise for each of the one or more regions of interest; determining one or more respective parameter values for the filter for each of the one or more regions of interest based on the respective noise model of the noise for each of the region of interests; and applying the filter with the one or more respective parameter values to a corresponding region of interest in the in focus image. ### Claim 15 (depends on 11) The method of claim 11, wherein estimating the amount of noise in the out of focus image further comprises: dividing the out of focus image into a plurality of sections; applying a frequency transform to each of the plurality of sections to identify the one or more frequency components and the amplitudes of the noise in the respective section; and identifying the frequency spread of the one or more frequency components and the amplitudes of the noise in the respective section. ### Claim 16 (depends on 15) The method of claim 15, wherein generating the noise-reduced image further comprises: identifying for removal at least a portion of the frequency spread of the one or more frequency components and amplitudes of the noise in each of the plurality of sections; determining a respective one or more parameter values for the filter corresponding to at least the portion of the frequency spread of the one or more frequency components and amplitudes of the noise in each of the plurality of sections; and applying the filter with the respective one or more parameter values to at least a corresponding one of each of a plurality of sections of the in focus image. ### Claim 17 (depends on 11) The method of claim 11, wherein the filter with the at least one parameter value applies an inverse of at least a portion of noise components of the noise identified by the noise model. ### Claim 18 (depends on 11) The method of claim 11, wherein the in focus image and the out of focus image are captured in low light. ### Claim 19 (depends on 11) The method of claim 11, further comprising: estimating a second noise model of noise in at least a section of the in focus image corresponding to at least the section of the out of focus image, wherein the noise model of the out of focus image and the second noise model of the in focus image include frequency components; identifying a common set of frequency components in both the noise model of the out of focus image and the second noise model of the in focus image; wherein determining the at least one parameter value for the filter further comprises determining one or more parameter values based on the common set of frequency components; and wherein generating the noise-reduced image further comprises applying the filter with the one or more parameter values to the in focus image. ### Claim 20 (depends on 11) The method of claim 11, wherein an amount of noise removed from portions of the in focus image varies based on a measured noise for each portion of the in focus image. ### Claim 21 A non-transitory computer-readable medium instructions executable by a computer device, comprising: at least one instruction for causing the computer device to receive an in focus image and at least one out of focus image of a scene, wherein the out of focus image has a second value of at least one focus-related parameter offset from a first value of the at least one focus-related parameter used for the in focus image; at least one instruction for causing the computer device to estimate a noise model of noise in at least a section of the out of focus image based at least in part on identifying a frequency spread of one or more frequency components and amplitudes of the noise; at least one instruction for causing the computer device to determine at least one parameter value for a filter based on the noise model; and at least one instruction for causing the computer device to generate a noise-reduced image by applying the filter with the at least one parameter value to the in focus image to reduce at least part of the noise in at least a corresponding section of the in focus image. ### Claim 22 (depends on 21) The computer-readable medium of claim 21, further comprising: at least one instruction for causing the computer device to divide the out of focus image into a plurality of sections; at least one instruction for causing the computer device to apply a frequency transform to each of the plurality of sections to identify the one or more frequency components and the amplitudes of the noise in the respective section; at least one instruction for causing the computer device to identify the frequency spread of the one or more frequency components and the amplitudes of the noise in the respective section; at least one instruction for causing the computer device to identify for removal at least a portion of the frequency spread of the one or more frequency components and amplitudes of the noise in each of the plurality of sections; at least one instruction for causing the computer device to determine a respective one or more parameter values for the filter corresponding to at least the portion of the frequency spread of the one or more frequency components and amplitudes of the noise in each of the plurality of sections; and at least one instruction for causing the computer device to apply the filter with the respective one or more parameter values to at least a corresponding one of each of a plurality of sections of the in focus image.