# US 10,771,785 B2 · Image encoding apparatus, method of controlling the same, and storage medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,771,785 B2
- Document ID: US10771785B2
- Status: Granted
- Kind: B2
- Application number: 16/205,299
- Filed: 2018-11-30
- Published: 2019-06-13
- Granted: 2020-09-08
- Earliest claimed: 2017-12-08
- Assignees: Canon Kabushiki Kaisha
- Applicants: Canon Kabushiki Kaisha
- Inventors: Akihiro Oishi
- Web page: https://patentgrep.com/patent/US10771785B2

## CPC

- H04N 19/124: Quantisation
- H04N 19/176: the region being a block, e.g. a macroblock
- H04N 19/18: the unit being a set of transform coefficients
- H04N 19/186: the unit being a colour or a chrominance component
- H04N 19/1883: {the unit relating to sub-band structure, e.g. hierarchical level, directional tree, e.g. low-high [LH], high-low [HL], high-high [HH]}
- H04N 19/63: using sub-band based transform, e.g. wavelets
- H04N 19/65: using error resilience
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

An image encoding apparatus comprises a separation unit which separates a plurality of planes respectively configured in a single component for a frame of an inputted video; a transform unit which performs a wavelet transformation of a plane of interest in the plurality of planes; an extraction unit which, from each sub-band, extracts in order blocks representing the same region of an image; a quantizing unit which quantizes wavelet transformation coefficients; and an encoding unit which encodes the quantized wavelet transformation coefficients. Here, the quantizing unit includes a determination unit that, for each of the blocks, determines a correction parameter for correcting the quantization parameter based on a direct current value and an alternating current value of the block, and quantizes wavelet transformation coefficients in accordance with the corrected quantization parameter.

## Claims

### Claim 1

An image encoding apparatus for encoding a video, the apparatus comprising:
at least one memory storing instructions; and
at least one processor that implements the instructions to execute a plurality of tasks, including:
a separation task that separates a plurality of planes respectively configured in a single component for a frame of an input video;
a wavelet transform task that performs a wavelet transformation of a plane of interest in the plurality of planes obtained by the separation task;
an extraction task that, from each sub-band obtained by the wavelet transform task, extracts in order blocks representing the same region of an image;
a quantizing task that, using a quantization parameter, quantizes wavelet transformation coefficients for each of the blocks; and
an encoding tasks that encodes the wavelet transformation coefficients after the quantization by the quantizing task, wherein
the quantizing task:
determines, for each of the blocks, a correction parameter for correcting the quantization parameter based on a direct current value and an alternating current value of the block; and
quantizes the wavelet transformation coefficients in accordance with the quantization parameter resulting from correction by the determined correction.

### Claim 2 (depends on 1)

The apparatus according to claim 1, wherein the quantizing task:
selects, as the correction parameter to be used for correcting the quantization parameter for the block of interest, a correction parameter corresponding to a direct current value and an alternating current value of a block of interest that is a target of quantization from among a plurality of correction parameter; and
changes, in accordance with a state of a number of times that each correction parameter in the plurality of correction parameters was selected, an association relationship between a direct current value and an alternating current value and the plurality of correction parameters.

### Claim 3 (depends on 2)

The apparatus according to claim 2, wherein the quantizing task corrects, by the determined correction parameter, the quantization parameter determined in accordance with a code amount of the encoded data generated prior to the block of interest, and quantizes wavelet transformation coefficients in the block of interest in accordance with the quantization parameter resulting from the correction.

### Claim 4 (depends on 3)

The apparatus according to claim 3, wherein:
when a code amount of the encoded data of an i-th block Bi is represented by C(Bi):
a target code amount for the plane of interest is represented by T, a number of block included in the plane of interest is represented by N,
and the block of interest is the i-th block Bi, and the correction parameter, which is determined based on a direct current value and an alternating current value of the block of interest Bi, is represented by Px,
the quantizing task determines a quantization parameter Q for the block of interest Bi as:
Q=Qini+r×Σ{C(Bk)−T/N}+Px
wherein Σ represents a total of k=1, 2, . . . , i−1, Qini indicates an initial quantization parameter, and r indicates a coefficient representing a sensitivity set in advance.

### Claim 5 (depends on 4)

The apparatus according to claim 4, wherein
the quantizing task
obtains average values LLave, HLave, LHave, and HHave of wavelet transform coefficients of the block of interest in sub-bands LL, HL, LH, and HH, and determines in accordance with the following formulas a DC value indicating a direct current value and an AC value indicating an alternating current value for the block of interest:
DC=LLave; and
AC=max{HLave,LHave,HHave},
where max { . . . } is a function that returns a maximum value within the parentheses, and
one value among the plurality of values assigned to a two-dimensional coordinate space for direct current values and alternating current values set in advance is selected as Px, which is the correction parameter, based on coordinates that the DC value and the AC value represent.

### Claim 6 (depends on 5)

The apparatus according to claim 5, wherein:
the two-dimensional coordinate space is configured by n×m regions divided in accordance with a number of thresholds that is set in advance and following axial directions for direct current values and alternating current values, and
values assigned to the n×m regions are smaller than zero for smaller direct current values and smaller alternating current values, and larger than zero for larger direct current values and larger alternating current values.

### Claim 7 (depends on 6)

The apparatus according to claim 6, wherein the plurality of tasks include:
a judgement task that, when a plane of a same component as the plane of interest in a frame immediately preceding a frame that the plane of interest belongs to is assumed to be a preceding plane, determines whether or not in the preceding plane there was excessive correction, which represents when the number of times that a negative value is selected for the correction parameter exceeds a number of times set in advance, and a code amount of encoded data exceeds a target code amount; and
a correction task that, in a case where the judgement task determines that there was the excessive correction in encoding of the preceding plane, corrects a threshold for determining the n×m regions to be a smaller value.

### Claim 8 (depends on 7)

The apparatus according to claim 7, wherein the separation task, from an image of a Bayer array frame, separates a G1 plane, a G2 plane, an R plane, and a B plane, which are respective components of 2×2 pixels in the Bayer array.

### Claim 9 (depends on 7)

The apparatus according to claim 7, wherein the separation task, from G1, G2, R, and B values that 2×2 pixels in an image of a Bayer array frame represent, calculates one brightness component Y and three chrominance components C1, C2, and C3 in accordance with the following formulas, and thereby generates a Y plane, a C1 plane, a C2 plane, and a C3 plane:
Y=(R+B+G1+G2)/4;
C1=R−G1;
C2=B−G2; and
C3=(R+G1)/2−(B+G2)/2.

### Claim 10 (depends on 7)

The apparatus according to claim 7, wherein the plurality of tasks include a processing task that, in a case where a correction of the correction task is continuously performed over a number of frames set in advance, reduce the number of regions having a negative value out of the values assigned to the n×m regions.

### Claim 11

A method of controlling an image encoding apparatus for encoding a video, the method comprising:
separating a plurality of planes respectively configured in a single component for a frame of an input video;
performing a wavelet transformation of a plane of interest in the plurality of planes obtained by the separation;
from each sub-band obtained in the wavelet transformation, extracting in order blocks representing the same region of an image;
using a quantization parameter, quantizing wavelet transformation coefficients for each of the blocks; and
encoding the wavelet transformation coefficients after the quantizing,
wherein the quantizing:
determines, for each of the blocks, a correction parameter for correcting the quantization parameter based on a direct current value and an alternating current value of the block; and
quantizes the wavelet transformation coefficients in accordance with the quantization parameter resulting from correction by the determined correction parameter.

### Claim 12

A non-transitory computer-readable storage medium storing a program executable by a computer to perform a method of controlling an image encoding apparatus for encoding a video, the method comprising:
separating a plurality of planes respectively configured in a single component for a frame of an input video;
performing a wavelet transformation of a plane of interest in the plurality of planes obtained by the separation;
from each sub-band obtained in the wavelet transformation, extracting in order blocks representing the same region of an image;
using a quantization parameter, quantizing wavelet transformation coefficients for each of the blocks; and
encoding the wavelet transformation coefficients after the quantizing,
wherein quantizing,
determines, for each of the blocks, a correction parameter for correcting the quantization parameter is determined based on a direct current value and an alternating current value of the block; and
quantizes the wavelet transformation coefficients in accordance with the quantization parameter resulting from correction by the determined correction parameter.