# US 11,308,585 B2 · Noise reduction method for high dynamic range videos > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,308,585 B2 - Document ID: US11308585B2 - Status: Granted - Kind: B2 - Application number: 16/613,945 - Filed: 2017-12-05 - Published: 2021-09-16 - Granted: 2022-04-19 - Earliest claimed: 2017-12-05 - Assignees: Google Inc. - Applicants: Google LLC - Inventors: Neil Birkbeck; Balineedu Adsumilli; Mohammad Izadi - Web page: https://patentgrep.com/patent/US11308585B2 ## CPC - G06T 5/70: Denoising; Smoothing - G06T 2207/10016: Video; Image sequence - G06T 2207/10024: Color image - G06T 2207/20081: Training; Learning - G06T 2207/20084: Artificial neural networks [ANN] - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract A method for denoising video content includes identifying a first frame block of a plurality of frame blocks associated with a first frame of the video content. The method also includes determining an average intensity value for the first frame block. The method also includes determining a first noise model that represents characteristics of the first frame block. The method also includes generating a denoising function using the average intensity value and the first noise model for the first frame block. The method further includes denoising the plurality of frame blocks using the denoising function. ## Claims ### Claim 1 A method for denoising high dynamic range video content, the method comprising: identifying flat blocks of a plurality of frame blocks associated with a first frame of the high dynamic range video content, the plurality of frame blocks comprising the flat blocks and non-flat blocks, wherein the flat blocks have a spatial and temporal intensity variance that is less than a threshold; for respective flat blocks of the flat blocks: determining an average intensity value for the flat block; and determining a noise model that represents noise and non-noise characteristics of the flat block; generating a denoising function that correlates the average intensity values with the noise models; and denoising the plurality of frame blocks using the denoising function. ### Claim 2 (depends on 1) The method of claim 1, wherein each noise model includes a power spectral density. ### Claim 3 (depends on 1) The method of claim 1, wherein denoising the plurality of frame blocks includes: for respective non-flat blocks of the plurality of frame blocks, determining an average intensity value for the non-flat block; and for respective frame blocks of the plurality of frame blocks, estimating a noise model for the frame block using the average intensity value for the frame block as input to the denoising function; and attenuating, using a Wiener filter, a noise signal in the frame block that is included in the noise model. ### Claim 4 (depends on 1) The method of claim 1, wherein the high dynamic range video content is represented in an RGB video format. ### Claim 5 (depends on 1) The method of claim 1, wherein the noise model includes a power spectral density that provides energy of noise at different frequencies, and determining the denoising function includes fitting a polynomial function to respective elements of the noise model. ### Claim 6 (depends on 1) The method of claim 1, wherein determining the denoising function includes training a neural network, using the average intensity values and the noise models determined for the flat blocks, to map average intensity values to noise models. ### Claim 7 (depends on 1) The method of claim 1, wherein determining the denoising function includes identifying bins corresponding to a range of intensity values and mapping each noise model to a respective bin that corresponds to a range of intensity values that the respective average intensity value is within. ### Claim 8 (depends on 1) The method of claim 1, wherein the noise characteristics vary in strength and perceptibility in a non-linear fashion along a brightness spectrum of the high dynamic range video content. ### Claim 9 (depends on 1) The method of claim 1, wherein denoising the plurality of frame blocks includes: for respective non-flat blocks of the plurality of frame blocks, determining an average intensity value for the non-flat block; and for respective frame blocks of the plurality of frame blocks, estimating a noise model for the frame block using the average intensity value for the frame block as input to the denoising function; transforming the frame block to the frequency domain; attenuating, using a Wiener filter, a noise signal in the transformed frame block that is included in the noise model to obtain a filtered transformed frame block; and obtaining a denoised frame block from the filtered transformed frame block. ### Claim 10 An apparatus for denoising high dynamic range video content, the apparatus comprising: a memory; and a processor, wherein the memory includes instructions executable by the processor to: identify flat blocks of a plurality of frame blocks associated with a first frame of the high dynamic range video content, the plurality of frame blocks comprising the flat blocks and non-flat blocks; for respective flat blocks of the flat blocks: determine an average intensity value for the flat block; and determine a noise model that represents noise and non-noise characteristics of the flat block; generate a denoising function using that correlates the average intensity values with the noise models; and denoise the plurality of frame blocks using the denoising function. ### Claim 11 (depends on 10) The apparatus of claim 10, wherein the first noise model includes a power spectral density. ### Claim 12 (depends on 10) The apparatus of claim 10, wherein the memory further includes instructions executable by the processor to: for respective non-flat blocks of the plurality of frame blocks, determine an average intensity value for the non-flat block; and for respective frame blocks of the plurality of frame blocks, estimate a noise model for the frame block using the average intensity value for the frame block as input to the denoising function. ### Claim 13 (depends on 10) The apparatus of claim 10, wherein the high dynamic range video content is represented in a YUV video format. ### Claim 14 (depends on 10) The apparatus of claim 10, wherein the noise model includes a power spectral density that provides energy of noise at different frequencies, and the memory further includes instructions executable by the processor to determine the denoising function by fitting a polynomial function to respective elements of the noise model. ### Claim 15 (depends on 10) The apparatus of claim 10, wherein the memory further includes instructions executable by the processor to determine the denoising function by training a neural network, using the average intensity values and the noise models determined for the flat blocks, to map average intensity values to the noise models. ### Claim 16 (depends on 10) The apparatus of claim 10, wherein the memory further includes instructions executable by the processor to determine the denoising function by identifying bins corresponding to a range of intensity values and mapping each noise model to a respective bin that corresponds to a range of intensity values that the respective average intensity value is within. ### Claim 17 (depends on 10) The apparatus of claim 10, wherein to denoise the plurality of frame blocks includes to: for respective non-flat blocks of the plurality of frame blocks, determine an average intensity value for the non-flat block; and for respective frame blocks of the plurality of frame blocks: estimate a respective noise model for each the frame block of the plurality of frame blocks using the average intensity value for the frame block as input to the denoising function; and attenuate, using a Wiener filter, a noise signal in the frame block that is included in the noise model. ### Claim 18 (depends on 10) The apparatus of claim 10, wherein the noise characteristics vary in strength and perceptibility in a non-linear fashion along a brightness spectrum of the high dynamic range video content. ### Claim 19 (depends on 10) The apparatus of claim 10, wherein to denoise the plurality of frame blocks includes: for respective non-flat blocks of the plurality of frame blocks, determine an average intensity value for the non-flat block; and for respective frame blocks of the plurality of frame blocks, estimate a noise model for the frame block using the average intensity value for the frame block as input to the denoising function; transform the frame block to the frequency domain; attenuate, using a Wiener filter, a noise signal in the transformed frame block that is included in the noise model to obtain a filtered transformed frame block; and obtain a denoised frame block from the filtered transformed frame block. ### Claim 20 (depends on 10) The apparatus of claim 10, wherein to denoise the plurality of frame blocks includes to: for respective non-flat blocks of the plurality of frame blocks, determine an average intensity value for the non-flat block; and for respective frame blocks of the plurality of frame blocks: estimate a respective noise model for each the frame block of the plurality of frame blocks using the average intensity value for the frame block as input to the denoising function; and attenuate, in the frequency domain using a Wiener filter, a noise signal in the frame block that is included in the noise model.