# US 2015/0103900 A1 · Method and apparatus of inter-layer filtering for scalable video coding

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2015/0103900 A1
- Document ID: US20150103900A1
- Status: Published application
- Kind: A1
- Application number: 14/397,146
- Filed: 2013-05-21
- Published: 2015-04-16
- Earliest claimed: 2012-05-21
- Assignees: MEDIATEK Sionopolis Walk
- Applicants: MEDIATEK Sionopolis Walk
- Inventors: Shan Liu; Mei Guo; Chia-Yang Tsai; Ching-Yeh Chen; Shaw-Min Lei
- Web page: https://patentgrep.com/patent/US20150103900A1

## CPC

- H04N 19/31: in the temporal domain
- H04N 19/117: Filters, e.g. for pre-processing or post-processing (sub-band filter banks H04N19/635)
- H04N 19/14: Coding unit complexity, e.g. amount of activity or edge presence estimation (H04N19/146 takes precedence)
- H04N 19/154: Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion (use of rate-distortion criteria H04N19/147)
- H04N 19/176: the region being a block, e.g. a macroblock
- H04N 19/33: in the spatial domain
- H04N 19/503: involving temporal prediction (adaptive coding with adaptive selection between spatial and temporal predictive coding H04N19/107; adaptive coding with adaptive selection among a plurality of temporal predictive coding modes H04N19/109)
- H04N 19/59: involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
- H04N 19/80: Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation (H04N19/635, H04N19/86 take precedence)
- H04N 19/46: Embedding additional information in the video signal during the compression process (H04N19/517, H04N19/68, H04N19/70 take precedence)
- H04N 19/70: characterised by syntax aspects related to video coding, e.g. related to compression standards
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A method and apparatus for inter-layer prediction for scalable video coding are disclosed. Embodiments according to the present invention apply inter-layer adaptive filtering to the video data derived from the reconstructed BL video data to generate inter-layer adaptive filtered data. The inter-layer adaptive filtered data is then included as prediction data to encode or decode the EL video data. The video data derived from the reconstructed BL video data is up-sampled before applying inter-layer adaptive filtering. The up-sampling may also be included in the inter-layer adaptive filtering. In another embodiment, the inter-layer adaptive filtering comprises adaptive up-sampling. For up-sampled BL video data at locations not collocated with the EL video data, the up-sampled BL video data is divided into location types according to locations of the up-sampled BL video data. Each location type may have an individual filter for up-sampling video data in the group.

## Claims

### Claim 1

A method of inter-layer prediction for scalable video coding, wherein video data is configured into a Base Layer (BL) and an Enhancement Layer (EL) and wherein EL video data has higher spatial resolution or better video quality than BL video data, the method comprising:
receiving first data associated with the EL video data;
determining second data, wherein the second data is derived from reconstructed BL video data;
applying inter-layer adaptive filtering to the second data to generate inter-layer adaptive filtered data; and
applying predictive encoding or decoding to the first data based on prediction data, wherein the prediction data comprises the inter-layer adaptive filtered data.

### Claim 2 (depends on 1)

The method of claim 1, wherein the first data corresponds to the EL video data and, for scalable video encoding, the predictive encoding is applied to the EL video data.

### Claim 3 (depends on 1)

The method of claim 1, wherein the first data corresponds to prediction residue of the EL video data and, for scalable video decoding, the predictive decoding is applied to the prediction residue to recover the EL video data.

### Claim 4 (depends on 1)

The method of claim 1, wherein, when the EL video data has higher spatial resolution than the BL video data, the second data corresponds to up-sampled third BL data to match same spatial resolution of the EL video data and the third BL data is derived from the reconstructed BL video data.

### Claim 5 (depends on 1)

The method of claim 1, wherein the inter-layer adaptive filtering corresponds to inter-layer sample adaptive offset (IL-SAO), inter-layer adaptive loop filtering (IL-ALF), or a combination of IL-SAO and IL-ALF.

### Claim 6 (depends on 5)

The method of claim 5, wherein first filter information associated with the inter-layer adaptive filtering shares second filter information associated with base layer SAO, base layer ALF or a combination of the base layer SAO and the base layer ALF.

### Claim 7 (depends on 5)

The method of claim 5, wherein first filter information associated with the inter-layer adaptive filtering is shared by second filter information associated with enhancement layer SAO, enhancement layer ALF or a combination of the enhancement layer SAO and the enhancement layer ALF.

### Claim 8 (depends on 1)

The method of claim 1, wherein filter coefficients for the inter-layer adaptive filtering are determined to reduce difference between original EL video data and up-sampled third BL data according to a difference measure.

### Claim 9 (depends on 7)

The method of claim 7, wherein the filter coefficients for the inter-layer adaptive filtering are determined using mean square error (MSE) design procedure or Wiener filter design procedure.

### Claim 10 (depends on 1)

The method of claim 1, wherein the inter-layer adaptive filtering corresponds to a single filter or concatenated multiple filters, wherein if the EL video data has higher spatial resolution than the BL video data, the single filter or the concatenated multiple filters are determined based on original EL video data and either the second data or up-sampled third BL data, and the third BL data is derived from the reconstructed BL video data, and wherein if the EL video data has same spatial resolution as the BL video data, the single filter or the concatenated multiple filters are determined based on the original EL video data and the second data.

### Claim 11 (depends on 1)

The method of claim 1, wherein the prediction data further comprises up-sampled third data using a fixed interpolation filter, wherein the third data is derived from the reconstructed BL video data.

### Claim 12 (depends on 1)

The method of claim 1, wherein filter information associated with the inter-layer adaptive filtering is used by a picture and a flag is used by each region, slice, largest coding unit, coding unit, or block of the picture to indicate whether the inter-layer adaptive filtering is applied to the region, slice, largest coding unit, coding unit, or block of the picture.

### Claim 13 (depends on 1)

The method of claim 1, wherein the second data corresponds to the reconstructed BL video data or output data resulted from applying in-loop filtering, a combination of the in-loop filtering and sample adaptive offset (SAO), or a combination of the in-loop filtering, the SAO and adaptive loop filtering (ALF) to the reconstructed BL video data.

### Claim 14 (depends on 1)

The method of claim 1, wherein filter information associated with the inter-layer adaptive filtering is included in a bitstream on a block basis or quadtree basis.

### Claim 15 (depends on 1)

The method of claim 1, wherein filter information associated with the inter-layer adaptive filtering is included in a slice level or a largest coding unit (LCU) level.

### Claim 16 (depends on 1)

The method of claim 1, wherein the inter-layer adaptive filtering comprises adaptive up-sampling when the EL video data has higher spatial resolution than the BL video data, wherein the adaptive up-sampling is applied to the second data to up-sample the second data using adaptive filtering, and wherein the up-sampled second data has same spatial resolution as the EL video data.

### Claim 17 (depends on 16)

The method of claim 16, wherein, for the up-sampled second data at locations that collocates with the EL video data, the adaptive up-sampling operates on collocated second data to generate the inter-layer adaptive filtered data for the inter-layer prediction, wherein the collocated second data corresponds to the up-sampled second data at locations that collocate with the EL video data.

### Claim 18 (depends on 17)

The method of claim 17, wherein the up-sampled second data at locations that does not collocate with the EL video data are processed by the adaptive up-sampling after the up-sampled second data at locations that collocates with the EL video data are processed by the adaptive up-sampling.

### Claim 19 (depends on 16)

The method of claim 16, wherein the up-sampled second data that has no collocated pixels in the EL video data are classified into location types according to locations of the up-sampled second data, wherein each location type uses an individual up-sampling filter for the adaptive up-sampling of the up-sampled second data in the location type.

### Claim 20 (depends on 19)

The method of claim 19, wherein the adaptive up-sampling operates on the second data or filtered second data that collocates with the EL video data to generate interpolated data for the up-sampled second data that has no collocated pixels in the EL video data.

### Claim 21 (depends on 1)

The method of claim 1, wherein the inter-layer adaptive filtering is applied at a picture level or a region, wherein the region corresponds to a first group of coding units, a second group of largest coding units (LCUs), a slice, a third group of slices, a tile or a fourth group of tiles.

### Claim 22 (depends on 1)

The method of claim 1, wherein filter coefficients of the inter-layer adaptive filtering are included in picture parameter set (PPS), adaptive parameter set (APS), or a slice header.

### Claim 23 (depends on 1)

The method of claim 1, wherein a flag is included in each largest coding unit (LCU) to indicate whether the inter-layer adaptive filtering is applied to the LCU, wherein fixed filtering is used if the flag indicates that the inter-layer adaptive filtering is not applied to the LCU.

### Claim 24 (depends on 1)

The method of claim 1, wherein a first flag is included in sequence parameter set (SPS) of a sequence to indicate whether the inter-layer adaptive filtering is allowed for pictures in the sequence.

### Claim 25 (depends on 24)

The method of claim 24, wherein, when the first flag indicates that the inter-layer adaptive filtering is allowed for the pictures in the sequence, a second flag is included in picture parameter set (PPS) of the picture to indicate if the inter-layer adaptive filtering is allowed for the picture.

### Claim 26 (depends on 1)

The method of claim 1, wherein the second data corresponds to the reconstructed BL video data, and wherein the inter-layer adaptive filtering comprises a fixed set of filters followed by adaptive filtering.

### Claim 27 (depends on 26)

The method of claim 26, wherein for scalable video decoding, a single filter is applied to the reconstructed BL video data, the single filter is equivalent to combined filtering using the fixed set of filters and the adaptive filtering, and wherein filter coefficients of the adaptive filtering are derived from a bitstream.

### Claim 28 (depends on 1)

The method of claim 1, wherein the inter-layer adaptive filtering comprises iterative inter-layer sample adaptive offset (IT-IL-SAO).

### Claim 29 (depends on 1)

The method of claim 1, wherein the inter-layer adaptive filtering comprises adaptive filtering followed by fixed up-sampling.

### Claim 30 (depends on 1)

The method of claim 1, wherein the inter-layer adaptive filtering comprises multiple inter-layer filtering branches and one of the multiple inter-layer filtering branches is selected for the inter-layer adaptive filtering.

### Claim 31

An apparatus for inter-layer prediction for scalable video coding, wherein video data is configured into a Base Layer (BL) and an Enhancement Layer (EL) and wherein EL video data has higher spatial resolution or better video quality than BL video data, the apparatus comprising:
electronic circuits, wherein the electronic circuits are configured to
receive first data associated with the EL video data;
determine second data, wherein the second data is derived from reconstructed BL video data;
apply inter-layer adaptive filtering to the second data to generate inter-layer adaptive filtered data; and
apply predictive encoding or decoding to the first data based on prediction data, wherein the prediction data comprises the inter-layer adaptive filtered data.
