Method and apparatus for convolutional neural network-based video denoising

An image capture device comprising:

• an image sensor configured to capture a central frame, an input frame that is temporally precedent to the central frame, a second input frame that is temporally precedent to the input frame, a third input frame that is temporally subsequent to the central frame, and a fourth input frame that is temporally subsequent to the third input frame;
• a first denoiser configured to denoise the central frame, the input frame, and the second input frame, and output a first denoised frame;
• a second denoiser configured to denoise the central frame, the input frame, and the third input frame, and output a second denoised frame;
• a third denoiser configured to denoise the central frame, the third input frame, and the fourth input frame, and output a third denoised frame;
• and
• a fourth denoiser configured to denoise the first denoised frame, the second denoised frame, and the third denoised frame, and output a fourth denoised frame,
• wherein the first denoiser, the second denoiser, the third denoiser, and the fourth denoiser are each further configured to receive a noise map and denoise respective frames based on the noise map.

An image capture device comprising:

• an image sensor configured to:
• capture a first plurality of frames comprising a central frame, an input frame that is temporally precedent to the central frame, and a second input frame that is temporally precedent to the input frame,
• capture a second plurality of frames comprising the central frame, the input frame, and a third input frame that is temporally subsequent to the central frame, and
• capture a third plurality of frames comprising the central frame, the third input frame, and a fourth input frame that is temporally subsequent to the third input frame;
• and
• a processor comprising:
• a first denoising layer, the first denoising layer comprising:
• a first denoiser configured to denoise the first plurality of frames and output a first denoised frame,
• a second denoiser configured to denoise the second plurality of frames and output a second denoised frame, and
• a third denoiser configured to denoise the third plurality of frames and output a third denoised frame;
• and
• a second denoising layer comprising a fourth denoiser configured to output a denoised frame based on the first denoised frame, the second denoised frame, and the third denoised frame,
• wherein the first denoising layer and the second denoising layer comprise a multi-scale architecture, and wherein the multi-scale architecture includes a plurality of skip-connections that forward an output of an encoder layer directly to an input of a corresponding decoder layer.