# US 9,883,083 B2 · Processing prior temporally-matched frames in 3D-based video denoising > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,883,083 B2 - Document ID: US9883083B2 - Status: Granted - Kind: B2 - Application number: 14/992,341 - Filed: 2016-01-11 - Published: 2016-04-28 - Granted: 2018-01-30 - Earliest claimed: 2009-06-05 - Assignees: Cisco Technology, Inc. - Applicants: Cisco Technology, Inc. - Inventors: Joel W. Schoenblum; Larry S. McKinney; Kenneth W. Heinrich - Web page: https://patentgrep.com/patent/US9883083B2 ## CPC - H04N 5/21: Circuitry for suppressing or minimising disturbance, e.g. moiré or halo - G06T 5/50: using two or more images, e.g. averaging or subtraction - G06T 5/70: Denoising; Smoothing - H04N 19/132: Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking - H04N 19/51: Motion estimation or motion compensation - H04N 19/587: involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence - H04N 19/895: in combination with error concealment - G06T 2207/20182: Noise reduction or smoothing in the temporal domain; Spatio-temporal filtering - H04N 5/00: Details of television systems (scanning details or combination thereof with generation of supply voltages H04N3/00) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract In one system embodiment, an overlapped block processing module configured to provide three-dimensional (3D) denoising of plural frames corresponding to a raw video sequence; and a frame alignment module configured to represent the raw video sequence with motion compensated frames corresponding to the raw video sequence, the motion compensated frames consisting of the plural frames and fewer in quantity than the quantity of frames of the raw video sequence, the plural frames based on prior temporally matched frames corresponding to the raw video sequence. ## Claims ### Claim 1 A frame alignment module, comprising: a memory; one or more processors configured to execute instructions stored in the memory, the instructions comprising: summing a multiplied frame with a first delayed frame and a reference frame to provide a summation frame, wherein the first delayed frame comprises an estimate of at least one of a plurality of first plural frames from another of the plurality of first plural frames; delaying the summation frame to provide a delayed summation frame wherein delaying comprises an imposition of a defined delay that varies based on an inputted frame; and frame matching the delayed summation frame with the reference frame to provide a second summation frame, wherein the frame matching comprises processing of luminance and chrominance data of the plurality of first plural frames. ### Claim 2 (depends on 1) The frame alignment module of claim 1, wherein the instructions further comprise: outputting the reference frame and the second summation frame by the frame alignment module for further processing. ### Claim 3 (depends on 1) The frame alignment module of claim 1, wherein each output of the frame matching comprises a set of motion vectors representing a block of pixels in one of a plurality of inputted frames, the block of pixels approximating a block of pixels in another of the plurality of inputted frames. ### Claim 4 (depends on 1) The frame alignment module of claim 1, wherein the summation frame comprises an estimate of the reference frame from the plurality of first plural frames. ### Claim 5 (depends on 1) The frame alignment module of claim 1, wherein the second summation frame comprises an estimate of the reference frame from the first summation frame delayed in time. ### Claim 6 (depends on 2) The frame alignment module of claim 2, wherein the instructions further comprise: outputting to a three-dimensional (3D) denoising loop, the 3D denoising loop subjecting the outputted frames to overlapped block processing. ### Claim 7 (depends on 2) The frame alignment module of claim 2, wherein the outputted frames comprise an approximation of a received raw video sequence, wherein frames of the received raw video sequence number greater in quantity than the outputted frames. ### Claim 8 (depends on 7) The frame alignment module of claim 7, wherein the processing comprises motion estimation and motion compensation. ### Claim 9 A frame alignment module, comprising: a memory; one or more processors configured to execute instructions stored in the memory, the instructions comprising: providing a summation frame comprised by summing a multiplied frame with a first delayed frame and a reference frame to provide the summation frame; delaying the summation frame by the frame alignment module to provide a delayed summation frame wherein delaying comprises an imposition of a defined delay that varies based on an inputted frame; and frame matching the delayed summation frame by the frame alignment module with the reference frame to provide a second summation frame, wherein the frame matching comprises processing of luminance and chrominance data of inputted frames, wherein one or more outputted frames from the second summation frame comprise an approximation of a received raw video sequence, wherein frames of the received raw video sequence number greater in quantity than the one or more outputted frames. ### Claim 10 (depends on 9) The frame alignment module of claim 9, wherein the instructions further comprise: outputting to a three-dimensional (3D) denoising loop, the 3D denoising loop subjecting the one or more outputted frames to overlapped block processing. ### Claim 11 (depends on 9) The frame alignment module of claim 9, wherein each of the one or more outputted frames comprises a set of motion vectors representing a block of pixels in one of a plurality of inputted frames, the block of pixels approximating a block of pixels in another of the plurality of inputted frames. ### Claim 12 (depends on 9) The frame alignment module of claim 9, wherein the summation frame comprises an estimate of at least one of plural frames from the another of the inputted frames. ### Claim 13 (depends on 9) The frame alignment module of claim 9, wherein the delayed summation frame comprises an estimate of the reference frame from the from the second summation frame. ### Claim 14 (depends on 9) The frame alignment module of claim 9, wherein the delayed summation frame comprises an estimate of the reference frame from the summation frame delayed in time. ### Claim 15 A system, comprising: a memory; one or more processors configured to execute instructions stored in the memory, the instructions comprising: receiving a first temporal sequence of first plural frames and a reference frame, at least a portion of the first plural frames and the reference frame corrupted by noise; frame matching the first plural frames; summing the frame-matched first plural frames with at least one of the first plural frames to provide a first summation frame; delaying the frame-matched first plural frames to provide a first delayed frame; frame matching the first summation frame to the reference frame to provide a second summation frame; self-combining the second summation frame multiple times to provide a multiplied frame; summing the multiplied frame with the first delayed frame and the reference frame to provide a third summation frame; delaying the third summation frame to provide a delayed third summation frame, wherein each of the delaying comprises an imposition of a defined delay that varies based on the inputted frame; frame matching the delayed third summation frame with the reference frame to provide a fourth summation frame, wherein each of the frame matching comprises processing of luminance and chrominance data of the inputted frames; and outputting the first delayed frame, the second summation frame, the reference frame, and the fourth summation frame for further processing. ### Claim 16 (depends on 15) The system of claim 15, wherein the outputted frames comprise an approximation of a received raw video sequence, the frames of the received raw video sequence greater in quantity than the outputted frames. ### Claim 17 (depends on 15) The system of claim 15, wherein each of the outputs of the frame matching comprises a set of motion vectors representing a block of pixels in one of the inputted frames, the block of pixels approximating a block of pixels in the other of the inputted frames. ### Claim 18 (depends on 15) The system of claim 15, wherein the frame-matched first plural frames comprises an estimate of the at least one of the first plural frames from another of the first plural frames. ### Claim 19 (depends on 15) The system of claim 15, wherein the second summation frame comprises an estimate of the reference frame from the first plural frames. ### Claim 20 (depends on 15) The system of claim 15, wherein the fourth summation frame comprises an estimate of the reference frame from the third summation frame delayed in time.