Deep Neural Network (DNN)-Based Reconstruction Method and Apparatus for Compressive Video Sensing (CVS)

A DNN-based apparatus for CVS, comprising:

• a video sequence input module, a non-key-frame reconstruction module, a GOP division module, a key-frame reconstruction module, and a video signal reconstruction and output module, wherein
• the video sequence input module is configured to input a video sequence;
• the GOP division module is configured to divide the video signal sequence input by the video sequence input module into a plurality of GOPs, and determine a key frame and a non-key frame for each GOP;
• the non-key-frame reconstruction module is configured to reconstruct the non-key frame obtained by the GOP division module through division, to obtain a reconstructed non-key frame;
• the key-frame reconstruction module is configured to reconstruct, by using a CVS reconstruction method, the key frame obtained by the GOP division module through division, to obtain a reconstructed key frame;
• the video signal reconstruction and output module is configured to arrange, in sequence, the reconstructed key frame output by the key-frame reconstruction module and the reconstructed non-key frame output by a DNN, to obtain a reconstructed video signal;
• the non-key-frame reconstruction module comprises an image block division module, an adaptive sampling module, a multi-hypothesis prediction module, a residual reconstruction module, a sub-network training module, and a DNN training module, wherein the adaptive sampling module, the multi-hypothesis prediction module, and the residual reconstruction module constitute the end-to-end DNN;
• the video sequence input module is configured to input a training-specific video sequence;
• the GOP division module is configured to divide the training-specific video sequence input by the video sequence input module into a plurality of GOPs, and determine a key frame and a non-key frame for each GOP;
• the image block division module is configured to divide the non-key frame that is of the training-specific video sequence and that is obtained by the GOP division module into a plurality of image blocks that do not overlap each other;
• the adaptive sampling module is configured to perform adaptive sampling on each image block in the non-key frame, to obtain measurements of the image block;
• the multi-hypothesis prediction module is configured to determine a hypothesis block corresponding to each image block, perform adaptive sampling on each image block in the non-key frame by using a corresponding hypothesis block, to obtain measurements of the hypothesis block, and obtain a multi-hypothesis predicted image based on the measurements of the hypothesis block and the measurements of the image block;
• the residual reconstruction module is configured to perform adaptive sampling on the multi-hypothesis predicted image, construct residual measurements between measurements of the multi-hypothesis predicted image and measurements of the non-key frame, and obtain a reconstructed non-key frame based on the residual measurements by using a residual reconstruction method;
• the sub-network training module is configured to cascade the adaptive sampling module and the multi-hypothesis prediction module to construct a sub-network, and train the sub-network;
• and
• the DNN training module is configured to train the entire DNN by using, as initial values, parameters obtained through sub-network pre-training, to obtain a trained DNN.