Back to results
US 10,764,592 B2Granted
Inter-layer residual prediction
AssigneeIntel Corporation
InventorsWenhao Zhang, Yi-Jen Chiu, Lidong Xu, Yu Han, Zhipin Deng, Xiaoxia Cai
Life of this patent
Estimating the term…
Abstract
Systems, devices and methods are described including performing scalable video coding using inter-layer residual prediction. Inter-layer residual prediction in an enhancement layer coding unit, prediction unit, or transform unit may use residual data obtained from a base layer or from a lower enhancement layer. The residual may be subjected to upsample filtering and/or refinement filtering. The upsample or refinement filter coefficients may be predetermined or may be adoptively determined.
Claims30
A method, comprising:
- at a scalable video decoder:
- accessing, for a block of an enhancement layer (EL) frame of scalable video content, multiple co-located blocks in one of a lower level EL frame or a base layer (BL) frame;
- and
- performing inter-layer residual prediction for a first region of the block of the EL frame responsive to a first block of the co-located blocks in the lower level EL frame or base layer (BL) frame corresponding to the first region being an intra coded block, wherein only regions of the block of the EL frame corresponding to intra coded blocks of the multiple co-located blocks in the lower level EL frame or base layer (BL) frame are inter-layer residual predicted, and wherein performing inter-layer residual prediction for the first region of the block of the EL frame comprises:
- determining a residual from the first block of the co-located blocks in the lower level EL frame or the BL frame;
- applying an upsample filter and a refining filter to the residual to generate an upsampled and filtered residual;
- and
- determining a predicted residual for the block of the EL frame based at least in part on adding the upsampled and filtered residual to a second residual of the first region of the block of the EL frame.
5 more elements
At least one non-transitory machine-readable medium comprising a plurality of instructions that, in response to being executed on a computing device, cause the computing device to perform scalable video decoding by:
- accessing, for a block of an enhancement layer (EL) frame of scalable video content, multiple co-located blocks in one of a lower level EL frame or a base layer (BL) frame;
- and
- performing inter-layer residual prediction for a first region of the block of the EL frame responsive to a first block of the co-located blocks in the lower level EL frame or base layer (BL) frame corresponding to the first region being an intra coded block, wherein only regions of the block of the EL frame corresponding to intra coded blocks of the multiple co-located blocks in the lower level EL frame or base layer (BL) frame are inter-layer residual predicted, and wherein performing inter-layer residual prediction for the first region of the block of the EL frame comprises:
- determining a residual from the first block of the co-located blocks in the lower level EL frame or the BL frame;
- applying an upsample filter and a refining filter to the residual to generate an upsampled and filtered residual;
- and
- determining a predicted residual for the block of the EL frame based at least in part on adding the upsampled and filtered residual to a second residual of the first region of the block of the EL frame.
4 more elements
An apparatus, comprising:
- memory to store scalable video content;
- and
- circuitry to access, for a block of an enhancement layer (EL) frame of the scalable video content, multiple co-located blocks in one of a lower level EL frame or a base layer (BL) frame, to perform inter-layer residual prediction for a first region of the block of the EL frame responsive to a first block of the co-located blocks in the lower level EL frame or base layer (BL) frame corresponding to the first region being an intra coded block, wherein only regions of the block of the EL frame corresponding to intra coded blocks of the multiple co-located blocks in the lower level EL frame or base layer (BL) frame are inter-layer residual predicted, and wherein to perform inter-layer residual prediction for the first region of the block of the EL frame, the circuitry is to access the memory to obtain a residual, wherein the residual is from the first block of the co-located blocks in the lower level EL frame or the BL frame, to apply an upsample filter and a refining filter to the residual to generate an upsampled and filtered residual, and to determine a predicted residual for the block of the EL frame based on an addition of the upsampled and filtered residual to a second residual of the first region of the block of the EL frame.
A method, comprising:
- at a scalable video encoder:
- determining a coding mode of a block of an enhancement layer (EL) frame of scalable video content;
- determining whether to perform inter-layer prediction for the block of the EL frame;
- and
- when inter-layer prediction is to be performed:
- accessing, for the block, multiple co-located blocks in one of a lower level EL frame or a base layer (BL) frame;
- and
- performing inter-layer residual prediction for a first region of the block of the EL frame responsive to a first block of the co-located blocks in the lower level EL frame or base layer (BL) frame corresponding to the first region being an intra coded block, wherein only regions of the block of the EL frame corresponding to intra coded blocks of the multiple co-located blocks in the lower level EL frame or base layer (BL) frame are inter-layer residual predicted, and wherein performing inter-layer residual prediction for the first region of the block of the EL frame comprises:
- determining a residual from the first block of the co-located blocks in the lower level EL frame or the BL frame;
- applying an upsample filter and a refining filter to the residual to generate an upsampled and filtered residual;
- and
- determining a second residual for the first region of the block of the EL frame based on differencing the upsampled and filtered residual with a predicted residual of the first region of the block of the EL frame.
9 more elements
At least one non-transitory machine-readable medium comprising a plurality of instructions that, in response to being executed on a computing device, cause the computing device to perform scalable video encoding by:
- determining a coding mode of a block of an enhancement layer (EL) frame of scalable video content;
- determining whether to perform inter-layer prediction for the block of the EL frame;
- and
- when inter-layer prediction is to be performed:
- accessing, for the block, multiple co-located blocks in one of a lower level EL frame or a base layer (BL) frame;
- and
- performing inter-layer residual prediction for a first region of the block of the EL frame responsive to a first block of the co-located blocks in the lower level EL frame or base layer (BL) frame corresponding to the first region being an intra coded block, wherein only regions of the block of the EL frame corresponding to intra coded blocks of the multiple co-located blocks in the lower level EL frame or base layer (BL) frame are inter-layer residual predicted, and wherein performing inter-layer residual prediction for the first region of the block of the EL frame comprises:
- determining a residual from the first block of the co-located blocks in the lower level EL frame or the BL frame;
- applying an upsample filter and a refining filter to the residual to generate an upsampled and filtered residual;
- and
- determining a second residual for the first region of the block of the EL frame based on differencing the upsampled and filtered residual with a predicted residual of the first region of the block of the EL frame.
8 more elements
Dependencies are inferred from the claim text.