Video codec assisted real-time video enhancement using deep learning
AssigneeIntel Corporation
InventorsChen Wang, Ximin Zhang, Huan Dou, Yi-Jen Chiu, Sang-Hee Lee
Life of this patent
Estimating the term…
Abstract
Techniques related to accelerated video enhancement using deep learning selectively applied based on video codec information are discussed. Such techniques include applying a deep learning video enhancement network selectively to decoded non-skip blocks that are in low quantization parameter frames, bypassing the deep learning network for decoded skip blocks in low quantization parameter frames, and applying non-deep learning video enhancement to high quantization parameter frames.
Claims20
A system for providing adaptive enhancement video processing comprising:
- a memory to store at least a portion of a bitstream;
- and
- one or more processors coupled to the memory, the one or more processors to:
- decode the bitstream to generate a video frame comprising pixel values at a first resolution, wherein the video frame comprises first and second blocks of pixel values at the first resolution, and wherein the first and second blocks comprise first and second video coding modes, respectively;
- apply a deep learning network to the first block in response to the first coding mode to generate a first output block comprising pixel values at a second resolution greater than the first resolution;
- bypass application of the deep learning network to the second block and retrieve a prior second output block comprising pixel values at the second resolution for the second block in response to the second coding mode;
- and
- generate an output video frame comprising pixel values at the second resolution corresponding to the video frame based at least on a merge of the first and second output blocks into the output video frame.
A method for providing adaptive enhancement video processing comprising:
- decoding a bitstream to generate a video frame comprising pixel values at a first resolution, wherein the video frame comprises first and second blocks of pixel values at the first resolution, and wherein the first and second blocks comprise first and second video coding modes, respectively;
- applying a deep learning network to the first block in response to the first coding mode to generate a first output block comprising pixel values at a second resolution greater than the first resolution;
- bypassing application of the deep learning network to the second block and retrieving a prior second output block for the second block in response to the second coding mode;
- and
- generating an output video frame corresponding to the video frame based at least on merging the first and second output blocks into the output video frame.
At least one non-transitory machine readable medium comprising a plurality of instructions that, in response to being executed on a computing device, cause the computing device to provide adaptive enhancement video processing by:
- decoding a bitstream to generate of a video frame comprising pixel values at a first resolution, wherein the video frame comprises first and second blocks of pixel values at the first resolution, and wherein the first and second blocks comprise first and second video coding modes, respectively;
- applying a deep learning network to the first block in response to the first coding mode to generate a first output block comprising pixel values at a second resolution greater than the first resolution;
- bypassing application of the deep learning network to the second block and retrieving a prior second output block for the second block in response to the second coding mode;
- and
- generating an output video frame corresponding to the video frame based at least on merging the first and second output blocks into the output video frame.
Dependencies are inferred from the claim text.