# US 2011/0235932 A1 · Method and device for selecting optimal transform matrices for down-sampling dct image

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2011/0235932 A1
- Document ID: US20110235932A1
- Status: Published application
- Kind: A1
- Application number: 13/155,584
- Filed: 2011-06-08
- Published: 2011-09-29
- Earliest claimed: 2007-06-04
- Assignees: Research In Motion Limited
- Inventors: Xiang Yu; En-hui Yang; Haiquan Wang
- Web page: https://patentgrep.com/patent/US20110235932A1

## CPC

- G06T 3/4084: in the transform domain, e.g. fast Fourier transform [FFT] domain scaling
- H04N 19/40: using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
- H04N 19/48: using compressed domain processing techniques other than decoding, e.g. modification of transform coefficients, variable length coding [VLC] data or run-length data (motion estimation in a transform domain H04N19/547; processing of decoded motion vectors H04N19/513)
- H04N 19/60: using transform coding
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Down-sampling of an image may be performed in the DCT domain. Transform matrices are obtained for down-sampling a DCT image of size M×N to a down-sampled DCT image of size I×J. The transform matrices may be used to down-sample the DCT image directly in the DCT domain. A spatial domain down-sampling method is selected and applied to the DCT image to produce a down-sampled DCT reference image. The transform matrices are selected by solving an optimization problem, leading to transform matrices which achieve a desired trade-off between the visual quality of images obtained using the transform matrices and the computational complexity associated with using the transform matrices. The visual quality is a measure of the difference between the down-sampled DCT image obtained using the transform matrices and the visual quality of the DCT reference image obtained using a spatial domain down-sampling method.

## Claims

### Claim 1

A method of selecting transform matrices for down-sampling in the discrete cosine transform (DCT) domain, the method comprising:
converting a DCT reference image into a spatial domain reference image;
applying a spatial domain down-sampling method to the spatial domain reference image to produce a spatial domain down-sampled reference image;
converting the spatial domain down-sampled reference image to the DCT domain to produce a down-sampled DCT reference image; and
selecting transform matrices to minimize an optimization expression, wherein the expression is based on the visual quality of a down-sampled DCT image obtained by applying the transform matrices to the DCT reference image relative to the visual quality of the down-sampled DCT reference image, and based on the computational complexity associated with down-sampling using the transform matrices.

### Claim 2 (depends on 1)

The method of claim 1, wherein selecting transform matrices includes determining the visual quality of the down-sampled DCT image obtained by applying the transform matrices to the DCT reference image relative to the visual quality of the down-sampled DCT reference image by calculating the mean squared error between the down-sampled DCT image obtained using the transform matrices and the down-sampled DCT reference image.

### Claim 3 (depends on 1)

The method of claim 1, wherein the transform matrices comprise a row-transform matrix and a column-transform matrix, and wherein selecting transform matrices includes determining the visual quality of down-sampled DCT images obtained by applying the row-transform matrix and the column-transform matrix to the DCT reference image, and determining the computational complexity associated with applying the row-transform matrix and the column-transform matrix to down-sample the DCT reference image.

### Claim 4 (depends on 1)

The method of claim 1, wherein the transform matrices comprise a row-transform matrix and a column-transform matrix, and wherein selecting transform matrices includes selecting transform matrices that satisfy the optimization expression:
min
D
IM
,
W
NJ

(

D
IM
·
C
MN
·
W
NJ
-
V
IJ

2
+
λ
·
r
g
)
,
where DIM is the row-transform matrix; WNJ is the column-transform matrix; CMN is the DCT image; DIM·CMN·WNJ is the down-sampled DCT image obtained using the transform matrices; VIJ is the down-sampled DCT reference image; rg is a measure of the computational complexity associated with using the transform matrices DIM,WNJ to down-sample an image CMN; and λ is a quality-and-complexity trade-off parameter for balancing the trade-off between visual quality of the down-sampled DCT image and the computational complexity of obtaining the down-sampled DCT image.

### Claim 5 (depends on 1)

The method of claim 1 further comprising:
retrieving a quality-and-complexity trade-off parameter,
and wherein selecting transform matrices includes using the quality-and-complexity trade-off parameter for balancing visual quality with computational complexity.

### Claim 6 (depends on 1)

The method of claim 1, wherein the computational complexity is a function of the absolute values of elements in the transform matrices.

### Claim 7 (depends on 6)

The method of claim 6 further comprising calculating the summation of the absolute value of all elements in the transform matrices.

### Claim 8 (depends on 1)

The method of claim 1, wherein the computational complexity is a function of the absolute values of all elements in the transform matrices below a selective learning with forgetting threshold parameter.

### Claim 9 (depends on 8)

The method of claim 8 further comprising calculating the summation of the absolute values of all elements in the transform matrices below the selective learning with forgetting threshold parameter.

### Claim 10 (depends on 1)

The method of claim 1, wherein the computational complexity is a function of the number of non-zero elements in a binary representation of each element of the transform matrices.

### Claim 11 (depends on 1)

The method of claim 1, wherein the DCT image is a frame of a video.

### Claim 12

An electronic device comprising:
a memory; and
a processor connected to the memory for obtaining transform matrices for down-sampling in the discrete cosine transform (DCT) domain, the processor being configured to:
convert a DCT reference image into a spatial domain reference image;
apply a spatial domain down-sampling method to the spatial domain reference image to produce a spatial domain down-sampled reference image;
convert the spatial domain down-sampled reference image to the DCT domain to produce a down-sampled DCT reference image; and
select transform matrices to minimize an optimization expression, wherein the optimization expression is based on the visual quality of a down-sampled DCT image obtained by applying the transform matrices to the DCT reference image relative to the visual quality of the down-sampled DCT reference image, and based on the computational complexity associated with down-sampling using the transform matrices.

### Claim 13 (depends on 12)

The electronic device of claim 12, wherein the processor is further configured to store the transform matrices in the memory.

### Claim 14 (depends on 12)

The electronic device of claim 12, wherein selecting transform matrices includes determining the visual quality of the down-sampled DCT image obtained by applying the transform matrices to the DCT reference image relative to the visual quality of the down-sampled DCT reference image by calculating the mean squared error between the down-sampled DCT image obtained using the transform matrices and the down-sampled DCT reference image.

### Claim 15 (depends on 12)

The electronic device of claim 12 wherein the transform matrices comprise a row-transform matrix and a column-transform matrix and wherein the optimization expression is:
min
D
IM
,
W
NJ

(

D
IM
·
C
MN
·
W
NJ
-
V
IJ

2
+
λ
·
r
g
)
,
where DIM is the row-transform matrix; WNJ is the column-transform matrix; CMN is the DCT image; DIM·CMN·WNJ is the down-sampled DCT image obtained using the transform matrices; VIJ is the down-sampled DCT reference image; rg is a measure of the computational complexity associated with using the transform matrices DIM,WNJ to down-sample an image CMN; and λ is a quality-and-complexity trade-off parameter for balancing the trade-off between visual quality of the down-sampled DCT image and the computational complexity of obtaining the down-sampled DCT image.

### Claim 16 (depends on 12)

The electronic device of claim 12, wherein the processor is configured to: determine if there is low power usage on the electronic device and to select the transform matrices in response to determining that there is low processor usage.

### Claim 17 (depends on 12)

The electronic device of claim 12, wherein the electronic device is a server which may be accessed by one or more wireless devices.

### Claim 18 (depends on 12)

The electronic device of claim 12, wherein the memory further stores a quality-and-complexity trade-off parameter, and wherein the quality-and-complexity trade-off parameter is included in the optimization expression to balance the relative importance of visual quality and computational complexity.

### Claim 19 (depends on 12)

The electronic device of claim 12, wherein the computational complexity is a function of the absolute value of all elements in transform matrices.

### Claim 20 (depends on 12)

The electronic device of claim 12, wherein the processor is configured to ignore the computational complexity due to values in the transform matrices that are above a predetermined threshold.
