# US 10,504,211 B2 · Sample-based video denoising > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,504,211 B2 - Document ID: US10504211B2 - Status: Granted - Kind: B2 - Application number: 15/630,881 - Filed: 2017-06-22 - Published: 2018-09-13 - Granted: 2019-12-10 - Earliest claimed: 2017-03-10 - Assignees: Disney Enterprises, Inc. - Applicants: Disney Enterprises, Inc. - Inventors: Christopher Schroers; Henning Zimmer; Alexander Sorkine Hornung - Web page: https://patentgrep.com/patent/US10504211B2 ## CPC - G06T 5/70: Denoising; Smoothing - G06T 1/20: Processor architectures; Processor configuration, e.g. pipelining - G06T 5/20: using local operators - G06T 5/73: Deblurring; Sharpening - H04N 5/21: Circuitry for suppressing or minimising disturbance, e.g. moiré or halo - H04N 5/213: Circuitry for suppressing or minimising impulsive noise (for suppressing or minimising disturbance in image signal generation H04N23/81) - G06T 2207/10016: Video; Image sequence - G06T 2207/10024: Color image - G06T 2207/20076: Probabilistic image processing - G06T 2207/20172: Image enhancement details - G06T 2207/20182: Noise reduction or smoothing in the temporal domain; Spatio-temporal filtering - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract According to one implementation, a video processing system includes a computing platform having a hardware processor and a system memory storing a sample-based video denoising software code. The hardware processor executes the sample-based video denoising software code to receive a video sequence, and select a reference frame of the video sequence to denoise. For each pixel of the reference frame, the hardware processor executes the sample-based video denoising software code to map the pixel to a sample pixel in each of other frames of the video sequence, identify a first confidence value corresponding to each of the sample pixels based on the mapping, identify a second confidence value corresponding to each of the sample pixels based on the frame that includes the sample pixel, and denoise the pixel based on a weighted combination of the sample pixels determined using the first confidence values and the second confidence values. ## Claims ### Claim 1 A video processing system comprising: a computing platform including a hardware processor and a system memory; a sample-based video denoising software code stored in the system memory; the hardware processor configured to execute the sample-based video denoising software code to: receive a video sequence; select a reference frame of the video sequence to denoise; for each pixel of the reference frame: map the pixel to a sample pixel in each of a plurality of other frames of the video sequence, the plurality of other frames including earlier and later frames of the reference frame of the video sequence; identify a first confidence value corresponding to each of the sample pixels, based on the mapping; identify a second confidence value corresponding to each of the sample pixels, based on a frame of the plurality of other frames that includes the sample pixel; and denoise the pixel based on a weighted combination of the sample pixels determined using the first confidence values and the second confidence values. ### Claim 2 (depends on 1) The video processing system of claim 1, wherein the pixel has a first location in the reference frame, and wherein the sample pixels have respective second locations in the plurality of other frames. ### Claim 3 (depends on 1) The video processing system of claim 1, wherein the pixel of the reference frame is mapped to the sample pixel in each of the plurality of other frames based on an optical flow field from the reference frame to each of the plurality of other frames. ### Claim 4 (depends on 1) The video processing system of claim 1, wherein the first confidence value corresponding to each of the sample pixels is based on a color match between the pixel of the reference frame and the sample pixel. ### Claim 5 (depends on 1) The video processing system of claim 1, wherein the first confidence value is proportional to a color match between the pixel of the reference frame and the sample pixel. ### Claim 6 (depends on 1) The video processing system of claim 1, wherein the second confidence value corresponding to each of the sample pixels is based on a temporal separation of the frame including the sample pixel from the reference frame. ### Claim 7 (depends on 1) The video processing system of claim 1, wherein the second confidence value corresponding to each of the sample pixels is inversely proportional to a temporal separation of the frame including the sample pixel from the reference frame. ### Claim 8 A method for use by a video processing system including a computing platform having a hardware processor and a system memory storing a sample-based video denoising software code, the method comprising: receiving, using the hardware processor, a video sequence; selecting, using the hardware processor, a reference frame of the video sequence to denoise; for each pixel of the reference frame: mapping, using the hardware processor, the pixel to a sample pixel in each of a plurality of other frames of the video sequence, the plurality of other frames including earlier and later frames of the reference frame of the video sequence; identifying, using the hardware processor, a first confidence value corresponding to each of the sample pixels, based on the mapping; identifying, using the hardware processor, a second confidence value corresponding to each of the sample pixels, based on a frame of the plurality of other frames that includes the sample pixel; and denoising, using the hardware processor, the pixel based on a weighted combination of the sample pixels determined using the first confidence values and the second confidence values. ### Claim 9 (depends on 8) The method of claim 8, wherein the pixel has a first location in the reference frame, and wherein the sample pixels have respective second locations in the plurality of other frames. ### Claim 10 (depends on 8) The method of claim 8, wherein the pixel of the reference frame is mapped to the sample pixel in each of the plurality of other frames based on an optical flow field from the reference frame to each of the plurality of other frames. ### Claim 11 (depends on 8) The method of claim 8, wherein the first confidence value corresponding to each of the sample pixels is based on a color match between the pixel of the reference frame and the sample pixel. ### Claim 12 (depends on 8) The method of claim 8, wherein the first confidence value is proportional to a color match between the pixel of the reference frame and the sample pixel. ### Claim 13 (depends on 8) The method of claim 8, wherein the second confidence value corresponding to each of the sample pixels is based on a temporal separation of the frame including the sample pixel from the reference frame. ### Claim 14 (depends on 8) The method of claim 8, wherein the second confidence value corresponding to each of the sample pixels is inversely proportional to a temporal separation of the frame including the sample pixel from the reference frame. ### Claim 15 A computer-readable non-transitory medium having stored thereon instructions, which when executed by a hardware processor, instantiate a method comprising: receiving a video sequence; selecting a reference frame of the video sequence to denoise; for each pixel of the reference frame: mapping the pixel to a sample pixel in each of a plurality of other frames of the video sequence, the plurality of other frames including earlier and later frames of the reference frame of the video sequence; identifying a first confidence value corresponding to each of the sample pixels, based on the mapping; identifying a second confidence value corresponding to each of the sample pixels, based on a frame of the plurality of other frames that includes the sample pixel; and denoising the pixel based on a weighted combination of the sample pixels determined using the first confidence values and the second confidence values. ### Claim 16 (depends on 15) The computer-readable non-transitory medium of claim 15, wherein the pixel has a first location in the reference frame, and wherein the sample pixels have respective second locations in the plurality of other frames. ### Claim 17 (depends on 15) The computer-readable non-transitory medium of claim 15, wherein the pixel of the reference frame is mapped to the sample pixel in each of the plurality of other frames based on an optical flow field from the reference frame to each of the plurality of other frames. ### Claim 18 (depends on 15) The computer-readable non-transitory medium of claim 15, wherein the first confidence value corresponding to each of the sample pixels is based on a color match between the pixel of the reference frame and the sample pixel. ### Claim 19 (depends on 15) The computer-readable non-transitory medium of claim 15, wherein the second confidence value corresponding to each of the sample pixels is based on a temporal separation of the frame including the sample pixel from the reference frame. ### Claim 20 (depends on 15) The computer-readable non-transitory medium of claim 15, wherein the second confidence value corresponding to each of the sample pixels is inversely proportional to a temporal separation of the frame including the sample pixel from the reference frame.