System and method to use decoder information in video super resolution
AssigneeAlibaba Group Holding Limited
InventorsMinghai Qin, Tae Meon Bae, Guanlin Wu, Yen-kuang Chen
Life of this patent
Estimating the term…
Abstract
A system for using decoder information in video super resolution processing. A compressed video buffering module is used for receiving a compressed video stream and a decoder module is used for decoding the compressed video stream into an uncompressed stream and extracting motion vector information from the uncompressed stream. A video super resolution deep neural network processor module is used for processing the uncompressed stream in conjunction with the motion vector information to produce a video super resolution stream. An output buffer module is used for buffering the video super resolution stream for subsequent output.
Claims20
A system for video super resolution processing;
- comprising:
- a decoder to decode a compressed video stream into an uncompressed stream and extract motion vector information from the uncompressed stream;
- a video buffer to receive, when the motion vector information is not available, the uncompressed stream from the decoder;
- a motion vector-based reconstruction processor to perform, when the motion vector information is available, motion vector-based video reconstruction based on the motion vector information received from the decoder, to generate motion vector-based reconstructed frames;
- a frame-based reconstruction processor to perform, when the motion vector information is not available, frame-based video reconstruction based on frames in the uncompressed stream received from the video buffer, to generate frame-based reconstructed frames;
- and
- a video super resolution deep neural network processor to receive the motion vector-based reconstructed frames from the motion vector-based reconstruction processor and the frame-based reconstructed frames from the frame-based reconstruction processor, and process the motion vector-based reconstructed frames and the frame-based reconstructed frames to produce a video super resolution stream.
A method for using decoder information in video super resolution processing, comprising:
- using a decoder to decode a compressed video stream into an uncompressed stream and extract motion vector information from the uncompressed stream;
- using a video buffer to receive, when the motion vector information is not available, the uncompressed stream from the decoder;
- using a motion vector-based reconstruction processor to perform, when the motion vector information is available, motion vector-based video reconstruction based on the motion vector information received from the decoder, to generate motion vector-based reconstructed frames;
- using a frame-based reconstruction processor to perform, when the motion vector information is not available, frame-based video reconstruction based on frames in the uncompressed stream received from the video buffer, to generate frame-based reconstructed frames;
- and
- using a video super resolution deep neural network processor to receive the motion vector-based reconstructed frames from the motion vector-based reconstruction processor and the frame-based reconstructed frames from the frame-based reconstruction processor, and process the motion vector-based reconstructed frames and the frame-based reconstructed frames to produce a video super resolution stream.
A device for using decoder information in video super resolution processing, comprising:
- a decoder to decode a compressed video stream into an uncompressed stream and extract motion vector information from the uncompressed stream;
- a video buffer to receive, when the motion vector information is not available, the uncompressed stream from the decoder;
- a motion vector-based reconstruction processor to perform, when the motion vector information is available, motion vector-based video reconstruction based on the motion vector information received from the decoder, to generate motion vector-based reconstructed frames;
- a frame-based reconstruction processor to perform, when the motion vector information is not available, frame-based video reconstruction based on frames in the uncompressed stream received from the video buffer, to generate frame-based reconstructed frames;
- and
- a video super resolution deep neural network processor to receive the motion vector-based reconstructed frames from the motion vector-based reconstruction processor and the frame-based reconstructed frames from the frame-based reconstruction processor, and process the motion vector-based reconstructed frames and the frame-based reconstructed frames to produce a video super resolution stream.
Dependencies are inferred from the claim text.