Video codec assisted real-time video enhancement using deep learning
AssigneeIntel Corporation
InventorsChen Wang, Ximin Zhang, Huan Dou, Yi-Jen Chiu, Sang-Hee Lee
Life of this patent
Estimating the term…
Abstract
Techniques related to accelerated video enhancement using deep learning selectively applied based on video codec information are discussed. Such techniques include applying a deep learning video enhancement network selectively to decoded non-skip blocks that are in low quantization parameter frames, bypassing the deep learning network for decoded skip blocks in low quantization parameter frames, and applying non-deep learning video enhancement to high quantization parameter frames.
Claims20
A method, comprising:
- determining whether a quantization parameter of a first image is smaller than a threshold, the first image having a first resolution and comprising a first pixel block and a second pixel block;
- in response to determining that the quantization parameter of the first image is smaller than the threshold:
- generating, by a deep learning network, a third pixel block from the first pixel block, and
- retrieving a fourth pixel block from a second image based on the second pixel block, wherein the fourth pixel block is a portion of the second image, and the second image is generated before the first image is generate;
- and
- generating a third image by merging the third pixel block and the fourth pixel block, wherein the third image has a second resolution that is higher than the first resolution.
One or more non-transitory computer-readable media storing instructions executable to perform operations, the operations comprising:
- determining whether a quantization parameter of a first image is smaller than a threshold, the first image having a first resolution and comprising a first pixel block and a second pixel block;
- in response to determining that the quantization parameter of the first image is smaller than the threshold:
- generating, by a deep learning network, a third pixel block from the first pixel block, and
- retrieving a fourth pixel block from a second image based on the second pixel block, wherein the fourth pixel block is a portion of the second image, and the second image is generated before the first image is generate;
- and
- generating a third image by merging the third pixel block and the fourth pixel block, wherein the third image has a second resolution that is higher than the first resolution.
An apparatus, comprising:
- a computer processor for executing computer program instructions;
- and
- a non-transitory computer-readable memory storing computer program instructions executable by the computer processor to perform operations comprising:
- determining whether a quantization parameter of a first image is smaller than a threshold, the first image having a first resolution and comprising a first pixel block and a second pixel block,
- in response to determining that the quantization parameter of the first image is smaller than the threshold:
- generating, by a deep learning network, a third pixel block from the first pixel block, and
- retrieving a fourth pixel block from a second image based on the second pixel block, wherein the fourth pixel block is a portion of the second image, and the second image is generated before the first image is generate, and
- generating a third image by merging the third pixel block and the fourth pixel block, wherein the third image has a second resolution that is higher than the first resolution.
Dependencies are inferred from the claim text.