# US 2013/0294523 A1 · Signal processing and tiered signal encoding

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2013/0294523 A1
- Document ID: US20130294523A1
- Status: Published application
- Kind: A1
- Application number: 13/188,207
- Filed: 2011-07-21
- Published: 2013-11-07
- Earliest claimed: 2011-07-21
- Inventors: Luca Rossato; Guido Meardi
- Web page: https://patentgrep.com/patent/US20130294523A1

## CPC

- H04N 19/59: involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
- H04N 19/105: Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
- H04N 19/117: Filters, e.g. for pre-processing or post-processing (sub-band filter banks H04N19/635)
- H04N 19/134: characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N 19/182: the unit being a pixel
- H04N 19/187: the unit being a scalable video layer
- H04N 19/33: in the spatial domain
- H04N 19/436: using parallelised computational arrangements
- H04N 19/82: involving filtering within a prediction loop
- H04N 19/85: using pre-processing or post-processing specially adapted for video compression
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

An encoder receives a signal. The encoder utilizes one or more downsample operations to produce downsampled renditions of the signal at successively lower levels of quality in the hierarchy. In a reverse direction, the encoder applies the one or more upsample operations to a downsampled rendition of the signal at a first level of quality to produce an upsampled rendition of the signal at a second level of quality in the hierarchy. The second level of quality is higher than the first level of quality. The one or more upsample operations and one or more downsample operations can be asymmetrical with respect to each other. That is, the function applied during downsampling can differ from the function applied when upsampling. The encoder produces residual data indicating a difference between the downsampled rendition of the signal at the second level of quality and the upsampled rendition of the signal at the second level of quality.

## Claims

### Claim 1

A method of encoding at different levels of quality in a hierarchy, the method comprising:
having a signal as input;
utilizing at least one downsample operation to produce downsampled renditions of the signal at successively lower levels of quality in the hierarchy;
applying at least one upsample operation to a downsampled rendition of the signal at a first level of quality to produce an upsampled rendition of the signal at a second level of quality in the hierarchy, the second level of quality being higher than the first level of quality, the at least one upsample operation and at least one downsample operation possibly being asymmetrical with respect to each other; and
producing residual data indicating a difference between the downsampled rendition of the signal at the second level of quality and the upsampled rendition of the signal at the second level of quality.

### Claim 2 (depends on 1)

The method as in claim 1, wherein either the at least one downsample operation or the at least one upsample operation, or possibly both, are non-linear.

### Claim 3 (depends on 1)

The method as in claim 1 further comprising:
repeatedly adjusting settings of elements in the downsampled rendition of the signal at the first level of quality to reduce an entropy of the residual data needed to reconstruct the downsampled rendition of the signal at the second level of quality.

### Claim 4 (depends on 1)

The method as in claim 1 further comprising:
for each of multiple upsample options used to upsample the rendition of the signal at the first level of quality into the rendition of the signal at the second level of quality, repeatedly adjusting settings of elements in the downsampled rendition of the signal at the first level of quality to reduce an entropy of the residual data needed to reconstruct the downsampled rendition of the signal at the second level of quality; and
identifying which of the multiple upsample options and tweaks to the downsampled rendition of the signal at the first level of quality results in the substantially lowest entropy for the residual data.

### Claim 5 (depends on 1)

The method as in claim 1 further comprising:
implementing a continuous transform function to simulate a substantially rectangular function, the continuous transform function facilitating quantification of entropy associated with the residual data.

### Claim 6 (depends on 5)

The method as in claim 5, wherein the continuous transform function is a sigmoid function.

### Claim 7 (depends on 5)

The method as in claim 5, wherein implementing the continuous transform function to simulate the substantially rectangular function further comprises:
generating an entropy function, a magnitude of the entropy function varying depending on different possible adjustments to the downsampled signal at the first level of quality; and
utilizing the entropy function to identify an adjustment to the downsampled signal at the first level of quality in which a respective entropy of the residual data for the adjustment is substantially minimal amongst the different possible adjustments.

### Claim 8 (depends on 1)

The method as in claim 1 further comprising:
parsing the downsampled rendition of the signal at the first level of quality into multiple regions of elements; and
simultaneously processing the multiple regions in parallel to identify adjustments to elements in the multiple regions to locally reduce an entropy associated with the residual data.

### Claim 9 (depends on 8)

The method as in claim 8 further comprising:
at least occasionally performing a global check of adjustments to the elements in the multiple regions to reduce an entropy of the residual data.

### Claim 10 (depends on 1)

The method as in claim 1 further comprising, wherein applying the at least one upsample operation includes:
for each of multiple elements in the rendition of the signal at the first level of quality, repeating steps of:
selecting an element to tweak in the downsampled signal at the first level of quality;
tweaking the selected element in different directions;
producing the residual data at the second level of quality based on the tweaked selected element; and
calculating an entropy of the residual data at the second level of quality to identify which tweak to the selected element at the first level of quality produces a set of residual data at the second level of quality having the substantially lowest entropy.

### Claim 11 (depends on 10)

The method as in claim 10, wherein calculating the entropy includes quantizing or simulating/approximating the quantization of the residual data to reduce the entropy.

### Claim 12 (depends on 10)

The method as in claim 10, wherein applying the at least one upsample operation further comprises applying multiple upsample operations to the downsampled rendition of the signal at the first level of quality; and
wherein producing the residual data includes identifying which of multiple upsample options applied to the downsampled rendition of the signal at the first level of quality produces residual data having the substantially lowest entropy.

### Claim 13 (depends on 1)

The method as in claim 1, wherein applying the at least one upsample operation further comprises:
applying multiple upsample operations to the downsampled rendition of the signal at the first level of quality; and
wherein producing the residual data includes identifying which of multiple upsample options applied to the downsampled rendition of the signal at the first level of quality produces residual data having the substantially lowest entropy.

### Claim 14 (depends on 1)

The method as in claim 1 further comprising:
repeatedly adjusting settings of elements in the downsampled rendition of the signal at the first level of quality to produce a tweaked rendition of the signal at the first level of quality;
upsampling the tweaked rendition of the signal at the first level of quality to produce the upsampled rendition of the signal at the second level of quality;
applying the residual data to the upsampled rendition of the signal at the second level of quality to produce an intermediate rendition of the signal at the second level of quality;
tweaking elements in the intermediate rendition of the signal at the second level of quality to produce a tweaked rendition of the signal at the second level of quality; and
generating a second set of residual data based on a difference between the upsampled rendition of the signal at the second level of quality and the tweaked rendition of the signal at the second level of quality.

### Claim 15 (depends on 1)

The method as in claim 1, wherein utilizing the at least one downsample operation to produce downsampled renditions of the signal at successively lower levels of quality in the hierarchy further comprises:
for each of multiple successively lower levels of quality in the hierarchy, repeating steps of:
downsampling the signal;
selecting an element to tweak in the downsampled signal at a selected level of quality;
tweaking the selected element in different directions to produce the residual data; and
calculating an entropy of the residual data to identify which tweak to the selected element produces a set of residual data having the substantially lowest entropy.

### Claim 16 (depends on 1)

The method as in claim 1, wherein utilizing the at least one downsample operation to produce the downsampled renditions of the signal at successively lower levels of quality further comprises:
selecting a level of quality J in which to downsample the signal, the level of quality J representing a highest level of quality in the hierarchy;
applying a selected downsampling operation to the signal at the level of quality J into a rendition of the signal at a level of quality J-1, the level of quality J-1 being a next lower level of quality beneath the level of quality J;
repeatedly tweaking settings of values in the rendition of the signal at level of quality J-1 to identify a tweaked rendition of the signal at the level of quality J-1 that reduces an entropy of residual data needed to reconstruct the signal at level of quality J.

### Claim 17 (depends on 16)

The method as in claim 16 further comprising:
applying a selected downsampling operation to the tweaked rendition of the signal at the level of quality J-1 into a rendition of the signal at a level of quality J-2, the level of quality J-2 being a next lower level of quality beneath the level of quality J-1;
repeatedly tweaking settings of values in the rendition of the signal at level of quality J-2 to identify a tweaked rendition of the signal at the level of quality J-2 that reduce an entropy of residual data needed to reconstruct the tweaked rendition of the signal at level of quality J-1; and
repeating steps of applying a selected downsampling operation and repeatedly tweaking settings in the hierarchy at pairs of successively lower levels of quality to produce the downsampled rendition of the signal at the second level of quality and the first level of quality.

### Claim 18 (depends on 16)

The method as in claim 16 further comprising:
repeatedly testing which of multiple possible upsample operations and tweaked settings of values in the rendition of the signal at level of quality J-1 reduces an entropy of residual data needed to reconstruct the signal at level of quality J.

### Claim 19

Computer-readable storage hardware having instructions stored thereon, the instructions, when carried out by a processing device, causing the processing device to perform operations of:
receiving a signal;
utilizing at least one downsample operation to produce downsampled renditions of the signal at successively lower levels of quality in the hierarchy;
applying at least one upsample operation to a downsampled rendition of the signal at a first level of quality to produce an upsampled rendition of the signal at a second level of quality in the hierarchy, the second level of quality being higher than the first level of quality, the at least one upsample operation and at least one downsample operation possibly being asymmetrical with respect to each other; and
producing residual data indicating a difference between the downsampled rendition of the signal at the second level of quality and the upsampled rendition of the signal at the second level of quality.

### Claim 20

A computer system comprising:
a processor;
a memory unit that stores instructions associated with an application executed by the processor; and
an interconnect coupling the processor and the memory unit, enabling the computer system to execute the application and perform operations of:
having a signal as input;
utilizing at least one downsample operation to produce downsampled renditions of the signal at successively lower levels of quality in the hierarchy;
applying at least one upsample operation to a downsampled rendition of the signal at a first level of quality to produce an upsampled rendition of the signal at a second level of quality in the hierarchy, the second level of quality being higher than the first level of quality, the at least one upsample operation and at least one downsample operation possibly being asymmetrical with respect to each other; and
producing residual data indicating a difference between the downsampled rendition of the signal at the second level of quality and the upsampled rendition of the signal at the second level of quality.
