# US 10,123,050 B2 · Filtering video data using a plurality of filters

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,123,050 B2
- Document ID: US10123050B2
- Status: Granted
- Kind: B2
- Application number: 12/406,585
- Filed: 2009-03-18
- Published: 2010-01-14
- Granted: 2018-11-06
- Earliest claimed: 2008-07-11
- Assignees: QUALCOMM Incorporated
- Applicants: Marta Karczewicz; Wei-Jung Chien
- Inventors: Marta Karczewicz; Wei-Jung Chien
- Web page: https://patentgrep.com/patent/US10123050B2

## CPC

- H04N 19/117: Filters, e.g. for pre-processing or post-processing (sub-band filter banks H04N19/635)
- H04N 19/44: Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
- H04N 19/46: Embedding additional information in the video signal during the compression process (H04N19/517, H04N19/68, H04N19/70 take precedence)
- H04N 19/80: Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation (H04N19/635, H04N19/86 take precedence)
- H04N 19/85: using pre-processing or post-processing specially adapted for video compression
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Systems and methods of filtering video data using a plurality of filters are disclosed. In an embodiment, a method includes receiving and decoding a plurality of filters embedded in a video data bitstream at a video decoder. The method includes selecting, based on information included in the video data bitstream, a particular filter of the plurality of filters. The method further includes applying the particular filter to at least a portion of decoded video data of the video data bitstream to produce filtered decoded video data.

## Claims

### Claim 1

A method of decoding video data, the method comprising:
decoding video data of a unit of pixels of at least one frame, filter selection information, and a plurality of filters embedded in a video data bitstream at a video decoder, the decoding comprising reconstructing pixel values of the pixels of the unit of the at least one frame using one of intra-frame prediction or inter-frame prediction, the pixels comprising first and second pixels of the unit of the at least one frame;
calculating, at the video decoder, a characteristic of each reconstructed pixel value from the reconstructed pixel values of the first pixels of the unit;
selecting, at the video decoder, a first filter of the plurality of filters to apply to a first reconstructed pixel value from the reconstructed pixel values of the first pixels, wherein selecting the first filter includes comparing a characteristic of the first reconstructed pixel value of the first pixels to a first range of values, a second range of values, and a third range of values defined by the filter selection information, wherein the filter selection information indicates that the first filter is to be applied to reconstructed pixel values that have characteristics within the first range of values, and wherein the first filter is selected, according to the filter selection information, based on the characteristic of the first reconstructed pixel value being within the first range of values;
calculating, at the video decoder, a characteristic of each reconstructed pixel value from the reconstructed pixel values of the second pixels of the unit;
selecting, at the video decoder, a second filter of the plurality of filters to apply to a second reconstructed pixel value from the reconstructed pixel values of the second pixels, wherein selecting the second filter includes comparing a characteristic of the second reconstructed pixel value of the second pixels to the first range of values, the second range of values, and the third range of values defined by the filter selection information, wherein the filter selection information indicates that the second filter is to be applied to reconstructed pixel values that have characteristics within the second range of values, wherein the second filter is selected, according to the filter selection information, based on the characteristic of the second reconstructed pixel value being within the second range of values, and wherein the second filter is different than the first filter; and
applying, at the video decoder, the first filter to the first reconstructed pixel value of the first pixels and the second filter to the second reconstructed pixel value of the second pixels to produce filtered decoded pixel values of the first and second pixels of the unit, wherein filter coefficients of at least one of the first filter or the second filter are quantized to form quantized filter coefficients, and wherein the quantized filter coefficients are subsequently clipped to be within a range after quantization is performed.

### Claim 2 (depends on 1)

The method of claim 1, wherein at least one of the characteristic of the first reconstructed pixel value or the characteristic of the second reconstructed pixel value includes at least one variance value from a mean value.

### Claim 3 (depends on 1)

The method of claim 1, wherein at least one of the characteristic of the first reconstructed pixel value or the characteristic of the second reconstructed pixel value includes an absolute value of pixel differences.

### Claim 4 (depends on 1)

The method of claim 1, wherein at least one of the characteristic of the first reconstructed pixel value or the characteristic of the second reconstructed pixel value includes a gradient pixel value.

### Claim 5 (depends on 1)

The method of claim 1, wherein at least one of the characteristic of the first reconstructed pixel value or the characteristic of the second reconstructed pixel value includes a sharpness measure.

### Claim 6 (depends on 1)

The method of claim 1, wherein the range is of about 0 to about 2 raised to the nth power, wherein the range is divided into a number of intervals m, and wherein the number of intervals m is determined at least partially based on indices of the quantized filter coefficients of each of the filters of the plurality of filters.

### Claim 7 (depends on 6)

The method of claim 6, wherein a particular quantized filter coefficient is determined by decoding a variable length codeword indicating a particular interval of the number of intervals m that corresponds to a value of the particular quantized filter coefficient, and by decoding a fixed length codeword specifying the value of the particular quantized filter coefficient within the particular interval.

### Claim 8 (depends on 1)

The method of claim 1, wherein quantized filter coefficients of the first filter of the plurality of filters are used to predict quantized filter coefficients of the second filter of the plurality of filters.

### Claim 9 (depends on 1)

The method of claim 1, wherein decoding the plurality of filters comprises decoding one or more quantized filter coefficients of the plurality of filters embedded in the video data bitstream.

### Claim 10 (depends on 9)

The method of claim 9, wherein decoding the plurality of filters comprises decoding one of the one or more quantized filter coefficients and predicting another of the one or more quantized filter coefficients based on the decoded one of the one or more quantized filter coefficients.

### Claim 11 (depends on 1)

The method of claim 1, further comprising outputting the filtered decoded video data for display.

### Claim 12 (depends on 1)

The method of claim 1, wherein the unit of the at least one frame comprises a block of pixels of the at least one frame.

### Claim 13

An apparatus for decoding video data, the apparatus comprising:
a video decoding processor configured to decode a unit of pixels of at least one frame, filter selection information, and a plurality of filters embedded in a video data bitstream, the processor is configured to reconstruct pixel values of the pixels of the unit of the at least one frame using one of intra-frame prediction or inter-frame prediction, the pixels comprising first and second pixels of the unit of the at least one frame; and
a memory configured to store the plurality of filters;
wherein the video decoding processor is further configured to:
calculate a characteristic of each reconstructed pixel value from the reconstructed pixel values of the first pixels of the unit;
select a first filter of the plurality of filters to apply to a first reconstructed pixel value from the reconstructed pixel values of the first pixels, wherein selecting the first filter includes comparing a characteristic of the first reconstructed pixel value of the first pixels to a first range of values, a second range of values, and a third range of values defined by the filter selection information, wherein the filter selection information indicates that the first filter is to be applied to reconstructed pixel values that have characteristics within the first range of values, and wherein the first filter is selected, according to the filter selection information, based on the characteristic of the first reconstructed pixel value being within the first range of values;
calculate a characteristic of each reconstructed pixel value from the reconstructed pixel values of the second pixels of the unit;
select a second filter of the plurality of filters to apply to a second reconstructed pixel value from the reconstructed pixel values of the second pixels, wherein selecting the second filter includes comparing a characteristic of the second reconstructed pixel value of the second pixels to the first range of values, the second range of values, and the third range of values defined by the filter selection information, wherein the filter selection information indicates that the second filter is to be applied to reconstructed pixel values that have characteristics within the second range of values, wherein the second filter is selected, according to the filter selection information, based on the characteristic of the second reconstructed pixel value being within the second range of values, and wherein the second filter is different than the first filter; and
apply the first filter to the first reconstructed pixel value of the first pixels and the second filter to the second reconstructed pixel value of the second pixels to produce filtered decoded pixel values of the first and second pixels of the unit, wherein filter coefficients of at least one of the first filter or the second filter are quantized to form quantized filter coefficients, and wherein the quantized filter coefficients are subsequently clipped to be within a range after quantization is performed.

### Claim 14 (depends on 13)

The apparatus of claim 13, further comprising a display device configured to display the filtered decoded video data.

### Claim 15 (depends on 13)

The apparatus of claim 13, wherein to decode the plurality of filters, the video processor is configured to decode one or more quantized filter coefficients of the plurality of filters.

### Claim 16 (depends on 15)

The apparatus of claim 15, wherein to decode the plurality of filters, the video processor is configured to decode one of the one or more quantized filter coefficients and predict another of the one or more quantized filter coefficients based on the decoded one of the one or more quantized filter coefficients.

### Claim 17 (depends on 13)

The apparatus of claim 13, wherein the unit of the at least one frame comprises a block of pixels of the at least one frame.

### Claim 18

An integrated circuit comprising:
video decoding circuitry configured to:
decode a signal including video data of a unit of pixels of at least one frame, filter selection information, and a plurality of filters embedded in a video data bitstream, wherein to decode the unit of video data, the video decoding circuitry is configured to reconstruct pixel values of the pixels of the unit of the at least one frame using one of intra-frame prediction or inter-frame prediction, the pixels comprising first and second pixels of the unit of the at least one frame;
calculate a characteristic of each reconstructed pixel value from the reconstructed pixel values of the first pixels of the unit;
select a first filter of the plurality of filters to apply to a first reconstructed pixel value from the reconstructed pixel values of the first pixels, wherein selecting the first filter includes comparing a characteristic of the first reconstructed pixel value of the first pixels to a first range of values, a second range of values, and a third range of values defined by the filter selection information, wherein the filter selection information indicates that the first filter is to be applied to reconstructed pixel values that have characteristics within the first range of values, and wherein the first filter is selected, according to the filter selection information, based on the characteristic of the first reconstructed pixel value being within the first range of values;
calculate a characteristic of each reconstructed pixel value from the reconstructed pixel values of the second pixels of the unit;
select a second filter of the plurality of filters to apply to a second reconstructed pixel value from the reconstructed pixel values of the second pixels, wherein selecting the second filter includes comparing a characteristic of the second reconstructed pixel value of the second pixels to the first range of values, the second range of values, and the third range of values defined by the filter selection information, wherein the filter selection information indicates that the second filter is to be applied to reconstructed pixel values that have characteristics within the second range of values, wherein the second filter is selected, according to the filter selection information, based on the characteristic of the second reconstructed pixel value being within the second range of values, and wherein the second filter is different than the first filter; and
apply the first filter to the first reconstructed pixel value of the first pixels and the second filter to the second reconstructed pixel value of the second pixels to produce filtered decoded pixel values of the first and second pixels of the unit, wherein filter coefficients of at least one of the first filter or the second filter are quantized to form quantized filter coefficients, and wherein the quantized filter coefficients are subsequently clipped to be within a range after quantization is performed.

### Claim 19 (depends on 18)

The integrated circuit of claim 18, wherein at least one or more of the characteristic of the first reconstructed pixel value or the characteristic of the second reconstructed pixel value includes at least one or more of a variance value from a mean value, an absolute value of pixel differences, a gradient pixel value, or a sharpness measure.

### Claim 20 (depends on 18)

The integrated circuit of claim 18, wherein to decode the signal including the plurality of filters, the video decoding circuitry is configured to decode one or more quantized filter coefficients of the plurality of filters.

### Claim 21 (depends on 20)

The integrated circuit of claim 20, wherein to decode the signal including the plurality of filters, the video decoding circuitry is configured to decode one of the one or more quantized filter coefficients and predict another of the one or more quantized filter coefficients based on the decoded one of the one or more quantized filter coefficients.

### Claim 22 (depends on 18)

The integrated circuit of claim 18, wherein the unit of the at least one frame comprises a block of pixels of the at least one frame.

### Claim 23

A video decoder comprising:
means for decoding video data of a unit of pixels of at least one frame, filter selection information, and a plurality of filters embedded in a video data bitstream, the means for decoding comprising means for reconstructing pixel values of the pixels of the unit of the at least one frame using one of intra-frame prediction or inter-frame prediction, the pixels comprising first and second pixels of the unit of the at least one frame;
means for calculating, at the video decoder, a characteristic of each reconstructed pixel value from the reconstructed pixel values of the first pixels of the unit;
means for selecting, at the video decoder, a first filter of the plurality of filters to apply to a first reconstructed pixel value from the reconstructed pixel values of the first pixels, wherein selecting the first filter includes comparing a characteristic of the first reconstructed pixel value of the first pixels to a first range of values, a second range of values, and a third range of values defined by the filter selection information, wherein the filter selection information indicates that the first filter is to be applied to reconstructed pixel values that have characteristics within the first range of values, and wherein the first filter is selected, according to the filter selection information, based on the characteristic of the first reconstructed pixel value being within the first range of values;
means for calculating, at the video decoder, a characteristic of each reconstructed pixel value from the reconstructed pixel values of the second pixels of the unit;
means for selecting, at the video decoder, a second filter of the plurality of filters to apply to a second reconstructed pixel value from the reconstructed pixel values of the second pixels, wherein selecting the second filter includes comparing a characteristic of the second reconstructed pixel value of the second pixels to the first range of values, the second range of values, and the third range of values defined by the filter selection information, wherein the filter selection information indicates that the second filter is to be applied to reconstructed pixel values that have characteristics within the second range of values, wherein the second filter is selected, according to the filter selection information, based on the characteristic of the second reconstructed pixel value being within the second range of values, and wherein the second filter is different than the first filter; and
means for applying, at the video decoder, the first filter to the first reconstructed pixel value of the first pixels and the second filter to the second reconstructed pixel value of the second pixels to produce filtered decoded pixel values of the first and second pixels of the unit, wherein filter coefficients of at least one of the first filter or the second filter are quantized to form quantized filter coefficients, and wherein the quantized filter coefficients are subsequently clipped to be within a range after quantization is performed.

### Claim 24 (depends on 23)

The apparatus of claim 23, further comprising means for receiving the video data bitstream via a wireless transmission.

### Claim 25 (depends on 23)

The video decoder of claim 23, wherein the unit of the at least one frame comprises a block of pixels of the at least one frame.

### Claim 26

A non-transitory computer-readable medium having instructions stored thereon that, when executed, cause one or more processors to:
decode, at a video decoder, a unit of pixels of at least one frame, filter selection information, and a plurality of filters embedded in a video data bitstream at the video decoder, wherein to decode the unit of pixels the processor is configured to reconstruct pixel values of the pixels of the unit of the at least one frame using one of intra-frame prediction or inter-frame prediction, the pixels comprising first and second pixels of the unit of the at least one frame;
calculate, at the video decoder, a characteristic of each reconstructed pixel value from the reconstructed pixel values of the first pixels of the unit;
select, at the video decoder, a first filter of the plurality of filters to apply to a first reconstructed pixel value from the reconstructed pixel values of the first pixels, wherein selecting the first filter includes comparing a characteristic of the first reconstructed pixel value of the first pixels to a first range of values, a second range of values, and a third range of values defined by the filter selection information, wherein the filter selection information indicates that the first filter is to be applied to reconstructed pixel values that have characteristics within the first range of values, and wherein the first filter is selected, according to the filter selection information, based on the characteristic of the first reconstructed pixel value being within the first range of values;
calculate, at the video decoder, a characteristic of each reconstructed pixel value from the reconstructed pixel values of the second pixels of the unit;
select, at the video decoder, a second filter of the plurality of filters to apply to a second reconstructed pixel value from the reconstructed pixel values of the second pixels, wherein selecting the second filter includes comparing a characteristic of the second reconstructed pixel value of the second pixels to the first range of values, the second range of values, and the third range of values defined by the filter selection information, wherein the filter selection information indicates that the second filter is to be applied to reconstructed pixel values that have characteristics within the second range of values, wherein the second filter is selected, according to the filter selection information, based on the characteristic of the second reconstructed pixel value being within the second range of values, and wherein the second filter is different than the first filter; and
apply, at the video decoder, the first filter to the first reconstructed pixel value of the first pixels and the second filter to the second reconstructed pixel value of the second pixels to produce filtered decoded pixel values of the first and second pixels of the unit, wherein filter coefficients of at least one of the first filter or the second filter are quantized to form quantized filter coefficients, and wherein the quantized filter coefficients are subsequently clipped to be within a range after quantization is performed.

### Claim 27 (depends on 26)

The non-transitory computer-readable medium of claim 26, wherein at least one or more of the characteristic of the first reconstructed pixel value or the characteristic of the second reconstructed pixel value includes at least one or more of a variance value from a mean value, an absolute value of pixel differences, a gradient pixel value, or a sharpness measure.

### Claim 28 (depends on 26)

The non-transitory computer-readable medium of claim 26, wherein to decode the plurality of filters, the instructions cause the one or more processors to decode one or more quantized filter coefficients of the plurality of filters.

### Claim 29 (depends on 28)

The non-transitory computer-readable medium of claim 28 wherein to decode the plurality of filters, the instructions cause the one or more processors to decode one of the one or more quantized filter coefficients and predict another of the one or more quantized filter coefficients based on the decoded one of the one or more quantized filter coefficients.

### Claim 30 (depends on 26)

The non-transitory computer readable medium of claim 26, wherein the unit of the at least one frame comprises a block of pixels of the at least one frame.