# US 2010/0135393 A1 · Method and apparatus for encoding and/or decoding video data using enhancement layer residual prediction for bit depth scalability

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2010/0135393 A1
- Document ID: US20100135393A1
- Status: Published application
- Kind: A1
- Application number: 12/452,293
- Filed: 2007-06-27
- Published: 2010-06-03
- Earliest claimed: 2007-06-27
- Assignees: Thomson Licensing
- Inventors: Yong Ying Gao; Yu Wen Wu; Charles Chuanming Wang
- Web page: https://patentgrep.com/patent/US20100135393A1

## CPC

- H04N 19/61: in combination with predictive coding
- H04N 19/33: in the spatial domain
- H04N 19/503: involving temporal prediction (adaptive coding with adaptive selection between spatial and temporal predictive coding H04N19/107; adaptive coding with adaptive selection among a plurality of temporal predictive coding modes H04N19/109)
- H04N 19/59: involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A scalable video bitstream may have an H.264/AVC compatible base layer and a scalable enhancement layer, where scalability refers to color bit depth. The H.264/AVC scalability extension SVC provides also other types of scalability, e.g. spatial scalability where the number of pixels in BL and EL are different. According to the invention, BL information is upsampled in two logical steps, one being texture upsampling and the other being bit depth upsampling. Texture upsampling is a process that increases the number of pixels, and bit depth upsampling is a process that increases the number of values that each pixel can have, corresponding to the pixels color intensity. The upsampled BL data are used to predict the collocated EL. The BL information is upsampled at the encoder side and in the same manner at the decoder side, wherein the upsampling refers to spatial and bit depth characteristics.

## Claims

### Claim at position 1

1-10. (canceled)

### Claim 11

Method for encoding video data having a base layer and an enhancement layer, wherein the base layer has lower bit-depth resolution and lower spatial, temporal and/or SNR resolution than the enhancement layer, the method comprising the steps of
encoding base layer data on slice level, wherein first base layer slices are intra-coded and second base layer slices are inter-coded;
reconstructing the encoded base layer slices;
upsampling reconstructed base layer slices, wherein first predicted versions of the corresponding enhancement layer slices are obtained that have higher spatial, temporal or SNR resolution than the base layer slices;
generating for a first, intra-coded base layer slice a tone mapping table defining an individual mapping between an upsampled reconstructed first base layer slice and the corresponding original enhancement layer slice, wherein both slices are compared to each other and wherein no tone mapping tables are generated for second, inter-coded base layer slices;
bit-depth upsampling the first predicted version of the enhancement layer slice based on said tone mapping, wherein a second predicted version of the corresponding enhancement layer slice is obtained that has higher bit-depth resolution than the first predicted version of the enhancement layer slice;
generating an enhancement layer residual being the difference between the original enhancement layer slice and the corresponding second predicted version of the enhancement layer slice; and
encoding the enhancement layer residual into an enhancement layer slice, wherein the enhancement layer slice includes in its slice header its corresponding tone mapping table.

### Claim 12 (depends on 11)

Method according to claim 11, wherein for each Intra coded slice pair an individual tone mapping table is generated and encoded, and wherein no tone mapping tables are generated for Inter coded slice pairs.

### Claim 13 (depends on 11)

Method according to claim 11, wherein for an Intra-coded slice separate tone mapping tables are generated for luminance and chrominance channels.

### Claim 14 (depends on 11)

Method according to claim 11, wherein the step of upsampling reconstructed base layer slices comprises in the case of Inter coded base layer slices upsampling of reconstructed base layer residual data.

### Claim 15

Method for decoding video data having a base layer and an enhancement layer and being encoded on slice level, the method comprising the steps of
receiving enhancement layer information and base layer information;
extracting from the enhancement layer information tone mapping data from slice headers of intra-coded enhancement layer slices;
performing inverse quantization and inverse transformation on the received information, wherein inverse quantized and inverse transformed enhancement layer information comprises residual information;
reconstructing intra-coded base layer slices;
upsampling base layer information, wherein the number of pixels is increased and for the increased number of pixels the value depth per pixel is increased, wherein for intra-coded slices the reconstructed base layer slices and the extracted tone mapping data are used, and wherein predicted enhancement layer information is obtained; and
reconstructing from the predicted enhancement layer information and the inverse quantized and inverse transformed enhancement layer information reconstructed enhancement layer video information.

### Claim 16 (depends on 15)

Method according to claim 15, wherein said tone mapping data are extracted only from each slice header of the enhancement layer.

### Claim 17 (depends on 15)

Method according to claim 15, further comprising the steps of extracting from the received inter-coded enhancement layer information motion information and reconstructing enhancement layer video, wherein said enhancement layer motion information is used.

### Claim 18

Apparatus for encoding video data having a base layer and an enhancement layer, wherein the base layer has lower bit-depth resolution and lower spatial, temporal and/or SNR resolution than the enhancement layer, comprising
means for encoding base layer data on slice level, wherein first base layer slices are intra-coded and second base layer slices are inter-coded;
means for reconstructing the encoded base layer slices;
means for upsampling reconstructed base layer slices, wherein first predicted versions of the corresponding enhancement layer slices are obtained that have higher spatial, temporal or SNR resolution than the base layer slices;
means for generating for a first, intra-coded base layer slice a tone mapping table defining an individual mapping between an upsampled reconstructed first base layer slice and the corresponding original enhancement layer slice, wherein both slices are compared to each other and wherein no tone mapping tables are generated for second, inter-coded base layer slices;
means for bit-depth upsampling the first predicted version of the enhancement layer slice based on said tone mapping, wherein a second predicted version of the corresponding enhancement layer slice is obtained that has higher bit-depth resolution than the first predicted version of the enhancement layer slice;
means for generating an enhancement layer residual being the difference between the original enhancement layer slice and the corresponding second predicted version of the enhancement layer slice; and
means for encoding the enhancement layer residual into an enhancement layer slice, wherein the enhancement layer slice includes in its slice header its corresponding tone mapping table.

### Claim 19 (depends on 18)

Apparatus according to claim 18, further comprising look-up table means for storing the tone mapping table.

### Claim 20

Apparatus for decoding video data having a base layer and an enhancement layer and being encoded on slice level, comprising
means for receiving enhancement layer information and means for receiving base layer information;
means for extracting from the enhancement layer information tone mapping data from slice headers of intra-coded enhancement layer slices;
means for performing inverse quantization and inverse transformation on the received information, wherein inverse quantized and inverse transformed enhancement layer information comprises residual information;
means for reconstructing intra-coded base layer slices;
means for upsampling base layer information, wherein the number of pixels is increased and for the increased number of pixels the value depth per pixel is increased, wherein for intra-coded slices the reconstructed base layer slices and the extracted tone mapping data are used, and wherein predicted enhancement layer information is obtained; and
means for reconstructing from the predicted enhancement layer information and the inverse quantized and inverse transformed enhancement layer information reconstructed enhancement layer video information.

### Claim 21 (depends on 20)

Apparatus according to claim 20, further comprising look-up table means for storing the tone mapping table.
