# US 2020/0404266 A1 · Scalable video coding using derivation of subblock subdivision for prediction from base layer

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0404266 A1
- Document ID: US20200404266A1
- Status: Published application
- Kind: A1
- Application number: 16/871,866
- Filed: 2020-05-11
- Published: 2020-12-24
- Earliest claimed: 2012-10-01
- Assignees: Ge Video Compression, LLC
- Applicants: GE Video Compression, LLC
- Inventors: Tobias HINZ; Haricharan LAKSHMAN; Jan STEGEMANN; Philipp HELLE; Mischa SIEKMANN; Karsten SUEHRING; Detlev MARPE; Heiko SCHWARZ; Christian BARTNIK; Ali Atef Ibrahim KHAIRAT ABDELHAMID; Heiner KIRCHHOFFER; Thomas WIEGAND
- Web page: https://patentgrep.com/patent/US20200404266A1

## CPC

- H04N 19/70: characterised by syntax aspects related to video coding, e.g. related to compression standards
- H04N 19/105: Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
- H04N 19/107: between spatial and temporal predictive coding, e.g. picture refresh
- H04N 19/117: Filters, e.g. for pre-processing or post-processing (sub-band filter banks H04N19/635)
- H04N 19/119: Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks
- H04N 19/136: Incoming video signal characteristics or properties
- H04N 19/137: Motion inside a coding unit, e.g. average field, frame or block difference
- H04N 19/157: Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
- H04N 19/159: Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
- H04N 19/176: the region being a block, e.g. a macroblock
- H04N 19/18: the unit being a set of transform coefficients
- H04N 19/187: the unit being a scalable video layer
- H04N 19/198: {including smoothing of a sequence of encoding parameters, e.g. by averaging, by choice of the maximum, minimum or median value}
- H04N 19/30: using hierarchical techniques, e.g. scalability (H04N19/63 takes precedence)
- H04N 19/33: in the spatial domain
- H04N 19/48: using compressed domain processing techniques other than decoding, e.g. modification of transform coefficients, variable length coding [VLC] data or run-length data (motion estimation in a transform domain H04N19/547; processing of decoded motion vectors H04N19/513)
- H04N 19/503: involving temporal prediction (adaptive coding with adaptive selection between spatial and temporal predictive coding H04N19/107; adaptive coding with adaptive selection among a plurality of temporal predictive coding modes H04N19/109)
- H04N 19/52: by predictive encoding
- H04N 19/587: involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
- H04N 19/59: involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
- H04N 19/593: involving spatial prediction techniques
- H04N 19/61: in combination with predictive coding
- H04N 19/615: using motion compensated temporal filtering [MCTF]
- H04N 19/82: involving filtering within a prediction loop
- H04N 19/865: {with detection of the former encoding block subdivision in decompressed video}
- H04N 19/11: among a plurality of spatial predictive coding modes
- H04N 19/34: Scalability techniques involving progressive bit-plane based encoding of the enhancement layer, e.g. fine granular scalability [FGS]
- H04N 19/36: Scalability techniques involving formatting the layers as a function of picture distortion after decoding, e.g. signal-to-noise [SNR] scalability
- H04N 19/463: by compressing encoding parameters before transmission
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Scalable video coding is rendered more efficient by deriving/selecting a subblock subdivision to be used for enhancement layer prediction, among a set of possible subblock subdivisions of an enhancement layer block by evaluating the spatial variation of the base layer coding parameters over the base layer signal. By this measure, less of the signalization overhead has to be spent on signaling this subblock subdivision within the enhancement layer data stream, if any. The subblock subdivision thus selected may be used in predictively coding/decoding the enhancement layer signal.

## Claims

### Claim 1

A video decoder for decoding a video represented by a base layer signal and an enhancement layer signal, the video decoder comprising:
a first decoding unit including a base layer decoder configured to reconstruct, using a processor, a base layer signal based on transform decoding of a base layer residual signal from a coded data stream; and
a second decoding unit including an enhancement layer decoder configured to reconstruct, using the processor, an enhancement layer signal from the coded data stream at least by:
determining an inter-layer prediction signal based on a scalability refinement of the reconstructed base layer signal, and
predictively reconstructing the enhancement layer signal using the inter-layer prediction signal,
wherein the first and second decoding units are configured to reconstruct the base layer and enhancement layer signals, respectively, based on one of the following:
bypassing both deblocking filtering and in-loop filtering,
bypassing one, and using the other, of the deblocking filtering and the in-loop filtering, and
using both of the deblocking and the in-loop filtering.

### Claim 2 (depends on 1)

The video decoder of claim 1, wherein the first and second decoding units are configured to bypass or use one or both of the deblocking filter and the in-loop filtering based on side information in the coded data stream, wherein the side-information signals a combination of the deblocking filtering and the in-loop filtering used for each respective portion of the video.

### Claim 3 (depends on 1)

The video decoder of claim 1, wherein the first decoding unit is configured to reconstruct the base layer signal from the coded data stream based on block-based prediction coding.

### Claim 4 (depends on 1)

The video decoder of claim 1, wherein the scalability refinement includes bit-depth refinement, quality refinement or resolution refinement.

### Claim 5 (depends on 4)

The video decoder of claim 4, wherein the second decoding unit is configured to perform the scalability refinement by up-sampling the reconstructed base layer signal so as to achieve the resolution refinement.

### Claim 6 (depends on 4)

The video decoder of claim 4, wherein the second decoding unit is configured to perform the scalability refinement by tone-mapping from n bits to m bits with m>n so as to achieve the bit depth refinement.

### Claim 7 (depends on 1)

The video decoder of claim 1, wherein the second decoding unit is configured to perform the predictive reconstruction by
spatially or temporally predicting a portion of an enhancement layer signal, currently to be reconstructed, from an already reconstructed portion of the enhancement layer signal to acquire an enhancement layer internal prediction signal, and
combining the inter-layer prediction signal and the enhancement layer internal prediction signal to acquire the enhancement layer prediction signal.

### Claim 8 (depends on 7)

The video decoder of claim 7, wherein the second decoding unit is configured to perform the combining by forming, at a portion currently to be reconstructed, a weighted average of the inter-layer prediction signal and the enhancement layer internal prediction signal to acquire the enhancement layer prediction signal.

### Claim 9 (depends on 8)

The video decoder of claim 8, wherein weights at which the inter-layer prediction signal and the enhancement layer internal prediction signal contribute to the enhancement layer prediction signal vary over different spatial frequency components.

### Claim 10 (depends on 2)

The video decoder of claim 2, wherein the deblocking and/or in-loop filtering varies in transfer function according to the side information in the coded data stream.

### Claim 11

A non-transitory computer-readable medium for storing data associated with a video, comprising a data stream stored in the non-transitory computer-readable medium, the data stream comprising information related to an encoded represented by a base layer signal and an enhancement layer signal, wherein the data stream is decoded using a plurality of operations including:
reconstructing, using a processor, the base layer signal based on transform decoding of a base layer residual signal from the data stream; and
reconstructing, using the processor, the enhancement layer signal from the data stream at least by:
determining an inter-layer prediction signal based on a scalability refinement of the reconstructed base layer signal, and
predictively reconstructing the enhancement layer signal using the inter-layer prediction signal,
wherein the reconstructing of the base layer and enhancement layer signals, respectively, is based on one of the following:
bypassing both deblocking filtering and in-loop filtering,
bypassing one, and using the other, of the deblocking filtering and the in-loop filtering, and
using both of the deblocking and the in-loop filtering.

### Claim 12 (depends on 11)

The non-transitory computer-readable medium of claim 11, wherein the plurality of operations further include processing side information from the data stream, wherein the side-information signals a combination of the deblocking filtering and the in-loop filtering to be used for each respective portion of the video.

### Claim 13

A video encoder for encoding a video, comprising:
a first encoding unit including a base layer encoder configured to, using a processor, determine a base layer residual signal for a base layer of the video, and encode into a data stream a base layer signal based on transform decoding of the base layer residual signal; and
a second encoding unit including an enhancement layer encoder configured to encode, using the processor, an enhancement layer signal into the coded data stream at least by:
determining an inter-layer prediction signal based on a scalability refinement of the base layer signal, and
predictively encoding the enhancement layer signal using the inter-layer prediction signal,
wherein the first and second encoding units are configured to encode the base layer and enhancement layer signals, respectively, based on one of the following:
bypassing both deblocking filtering and in-loop filtering,
bypassing one, and using the other, of the deblocking filtering and the in-loop filtering, and
using both of the deblocking and the in-loop filtering.

### Claim 14 (depends on 13)

The video encoder of claim 13, wherein the first and second encoding units are configured to bypass or use one or both of the deblocking filter and the in-loop filtering based on side information in the coded data stream, wherein the side-information signals a combination of the deblocking filtering and the in-loop filtering used for each respective portion of the video.

### Claim 15 (depends on 13)

The video encoder of claim 13, wherein the first encoding unit is configured to encode the base layer signal based on block-based prediction coding.

### Claim 16 (depends on 13)

The video encoder of claim 13, wherein the scalability refinement includes bit-depth refinement, quality refinement or resolution refinement.

### Claim 17 (depends on 16)

The video encoder of claim 16, wherein the second encoding unit is configured to perform the scalability refinement by up-sampling the reconstructed base layer signal so as to achieve the resolution refinement.

### Claim 18 (depends on 16)

The video encoder of claim 16, wherein the second encoding unit is configured to perform the scalability refinement by tone-mapping from n bits to m bits with m>n so as to achieve the bit depth refinement.

### Claim 19 (depends on 13)

The video encoder of claim 13, wherein the second encoding unit is configured to perform the predictive encoding by
spatially or temporally predicting a portion of an enhancement layer signal, currently to be encoded, from an already encoded portion of the enhancement layer signal to acquire an enhancement layer internal prediction signal, and
combining the inter-layer prediction signal and the enhancement layer internal prediction signal to acquire the enhancement layer prediction signal.

### Claim 20 (depends on 19)

The video encoder of claim 19, wherein the second encoding unit is configured to perform the combining by forming, at a portion currently to be encoded, a weighted average of the inter-layer prediction signal and the enhancement layer internal prediction signal to acquire the enhancement layer prediction signal.
