US 8,559,511 B2Granted

Method and apparatus for video coding by ABT-based just noticeable difference model

AssigneeHong Kong Applied Science and Technology Research Institute Company Limited

InventorsKing Ngi Ngan, Lin Ma, Wai-Kuen Cham, Yu Liu

Life of this patent

Estimating the term…

Loading patent…

Abstract

The present invention relates to method and apparatus for video coding by ABT-based just noticeable difference (JND). For building the just noticeable difference model, spatial content information (SCI) is used to represent the spatial appearance similarity between one macroblock and its sub-blocks and the motion characteristic distance (MCD) is used to represent the motion characteristics similarity between one macroblock and its sub-blocks. For intra frames, the balance strategy based on the obtained SCI of the macroblock is used to generate the ABT-based JND model. For inter frames, the balanced strategy based on the obtained SCI and MCD of the macroblock is used to generate the ABT-based JND model. Using the ABT-based JND model, the residual coefficients for each block in a frame is filtered to obtain a reduced set of residual coefficients for transmission without degradation in visual quality.

Claims17

Dependent claims

A video coding apparatus, comprising:

a unit for providing a video signal;
a prediction mode determination unit that determines whether a frame in the video signal is an inter frame or an intra frame according to the prediction mode of the frame;
an adaptive block-size transform (ABT)-based just noticeable difference (JND) model generation unit that makes use of one or more performance properties of how humans perceive things visually for different block-size based transforms and computes block types, block categorizations and motion vectors for block of different sizes to generate an ABT-based JND model which contains one or more JND thresholds as determined by a psychophysical experiment based on the visual perception model;
and
an ABT-based JND filter that filters one or more residual coefficients by discarding one or more residual coefficients which are below the JND thresholds of the ABT-based JND model and subtracting the JND thresholds from one or more residual coefficients which are larger than the JND thresholds of the ABT-based JND model so that a signal with filtered residual coefficients is output;
wherein the one or more JND thresholds are thresholds over which an observer of the output signal with filtered residual coefficients starts to notice any distortion;
and
wherein the one or more JND thresholds are modeled by
T
basic
⁡
(
m
,
n
,
i
,
j
)
=
s
ϕ
i
⁢
ϕ
j
·
exp
⁡
(
c
⁢
⁢
ω
i
,
j
)
/
(
a
+
b
⁢
⁢
ω
i
,
j
)
γ
+
(
1
+
γ
)
·
cos
2
⁢
φ
ij
where:
Tbasic(m, n, i, j) is a basic JND threshold;
(m, n) denotes the position of DCT block in an image;
s denotes the summation effect factor;
φi and φj are discrete cosine transform (DCT) normalization factors;
φij denotes directional angle of corresponding DCT coefficient;
ωi,j is corresponding spatial frequency of DCT subband (i, j);
γ denotes the oblique effect factor;
and
a, b and c are selected so as to achieve a least mean squared error in fitting JND thresholds obtained from a psychophysical experiment with the values obtained by computing Tbasic(m, n, i, j).

71 more elements

A method for coding a video signal, comprising:

providing a video signal to be coded;
determining whether a frame in the video signal is an inter frame or an intra frame according to the prediction mode of the frame;
generating an adaptive block-size transform (ABT)-based just noticeable difference (JND) model by making use of one or more performance properties of how humans perceive things visually for different block-size based transforms and computes block types, block categorizations and motion vectors for block of different sizes to generate the ABT-based JND model which contains one or more JND thresholds as determined by a psychophysical experiment based on the visual perception model;
and
filtering one or more residual coefficients by discarding one or more residual coefficients which are below the ABT-based JND thresholds of the ABT-based JND model and subtracting the JND thresholds from one or more residual coefficients which are larger than the JND thresholds of the ABT-based JND model so that a signal with filtered residual coefficients is output;
wherein the one or more JND thresholds are thresholds over which an observer of the output signal with filtered residual coefficients starts to notice any distortion;
and
wherein the one or more JND thresholds are modeled by
T
basic
⁡
(
m
,
n
,
i
,
j
)
=
s
ϕ
i
⁢
ϕ
j
·
exp
⁡
(
c
⁢
⁢
ω
i
,
j
)
/
(
a
+
b
⁢
⁢
ω
i
,
j
)
γ
+
(
1
+
γ
)
·
cos
2
⁢
φ
ij
where:
Tbasic(m, n, i, j) is a basic JND threshold;
(m, n) denotes the position of DCT block in an image;
s denotes the summation effect factor;
φi and φj are discrete cosine transform (DCT) normalization factors;
φij denotes directional angle of corresponding DCT coefficient;
ωi,j is corresponding spatial frequency of DCT subband (i, j);
γ denotes the oblique effect factor;
and
a, b and c are selected so as to achieve a least mean squared error in fitting JND thresholds obtained from a psychophysical experiment with the values obtained by computing Tbasic(m, n, i, j).

71 more elements

Dependencies are inferred from the claim text.

Facts

Application: 12/750,401Filed 30 Mar 2010
Published: US 2011/0243228 A16 Oct 2011
Granted: 15 Oct 2013
Term ends: Not estimatedThe source records don’t support an estimate.

Classification8 codes

H04N 19/137
Incoming video signal characteristics or properties: motion inside a coding unit, e.g. average field, frame or block difference
H04N 19/119
Characterised by the element, parameter or selection affected or controlled by the adaptive coding: adaptive subdivision aspects, e.g. subdivision…
H04N 19/122
Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or…:…
H04N 19/14
Incoming video signal characteristics or properties: coding unit complexity, e.g. amount of activity or edge presence estimation