# US 8,279,918 B2 · Method and apparatus for motion compensated temporal filtering using residual signal clipping

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 8,279,918 B2
- Document ID: US8279918B2
- Status: Granted
- Kind: B2
- Application number: 11/182,582
- Filed: 2005-07-15
- Published: 2007-01-18
- Granted: 2012-10-02
- Earliest claimed: 2005-07-15
- Assignees: UTC Fire & Security Americas Corporation, Inc.
- Inventors: Diego Santa Cruz; Julien Reichel; Francesco Ziliani
- Web page: https://patentgrep.com/patent/US8279918B2

## CPC

- H04N 19/615: using motion compensated temporal filtering [MCTF]
- H04N 19/107: between spatial and temporal predictive coding, e.g. picture refresh
- H04N 19/61: in combination with predictive coding
- H04N 19/63: using sub-band based transform, e.g. wavelets
- H04N 19/13: Adaptive entropy coding, e.g. adaptive variable length coding [AVLC] or context adaptive binary arithmetic coding [CABAC]
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A method for temporal decomposition and reconstruction of an input video signal is disclosed. The method can use a prediction process and an update process in the framework of motion compensated temporal filtering (MCTF), the method can employ a block based video codec having an encoder and a decoder. A residual picture signal and a low-pass picture signal are received, the low-pass picture signal having a lesser precision than that of the residual picture signal, and in response to the residual picture signal being decoded in the decoder, the residual picture signal is clipped and reduced in precision, such that the low-pass picture signal, a reconstructed picture, and the residual picture signal have equal precisions. An inverse MCTF process, using a lifting scheme, can be performed on the clipped residual signal.

## Claims

### Claim 1

A method for temporal decomposition and reconstruction of an input video signal that uses a prediction process and an update process in framework of motion compensated temporal filtering (MCTF), the motion information used for the update process being derived from the motion information used for the prediction process, the method employing a block based video codec composed of an encoder and a decoder, the method comprising:
receiving a residual picture signal having a precision;
receiving a low-pass picture signal having a precision that is less than the precision of the residual picture signal;
in response to the residual picture signal being decoded in the decoder, clipping the residual picture signal such that the residual picture signal is reduced in precision, such that the low-pass picture signal, a reconstructed picture, and the clipped residual picture signal have equal precisions;
performing an inverse MCTF process, comprising a lifting scheme, on the clipped residual picture signal;
in response to the block based video codec selecting a block mode for each block, avoiding selecting a block mode in which a residual clipping would lead to breaking a perfect reconstruction property of the MCTF, thereby defining a constrained mode decision,
wherein the constrained mode decision considers a mode in accordance with the following:
define ĥc[x,k]=C(ĥ[x,k], where
dmax is used to determine whether to select a considered block mode,
x is a vector describing a position of a pixel in a picture,
k describes a temporal position of the picture,
ĥ[x,k] describes the residual picture signal,
C is a clipping function C(w),
ĥC[x,k] are values stored in a storage medium,
C
⁡
(
w
)
=
{
A
min
if
w
<
A
min
,
A
max
if
w
>
A
max
,
w
otherwise
,
Amin and Amax are minimum and maximum values, respectively, that may be represented with b bits of precision in signed form, and w is an attribute of clipping function C(w);
define
⁢
⁢
d
max
=
max
x
⁢
∈
⁢
B
⁢

C
′
(
⁢
h
~
⁡
[
x
,
⁢
k
]
)
⁢
-
⁢
h
~
⁡
[
x
,
⁢
k
]

,
where
B is a block being considered,
C′ is clipping function C modified such that A′min and A′max limits are used instead of Amin and Amax, respectively with A′min≧Amin and A′max≦Amax,
{tilde over (h)}[x,k] is an estimation of the signal ĥ[x,k] that will be received at the decoder; and
verify if dmax=0, and if not, forbid the considered mode from being selected for block B.

### Claim 2 (depends on 1)

The method of claim 1, wherein the clipping of the residual picture signal comprises:
clipping the residual picture signal to a b-bit precision just after the signal is decoded, wherein the b-bit precision is equal to or less than the bit precision of the received low-pass picture signal.

### Claim 3 (depends on 2)

The method of claim 2, wherein a value of b is 8.

### Claim 4 (depends on 1)

The method of claim 1, further comprising:
in response to anticipated dipping of the residual picture signal at the decoder, using the dipped residual picture signal at the encoder during an update process, thereby resulting in an encoder process being closely matched to a decoder process.

### Claim 5 (depends on 4)

The method of claim 4, wherein:
in response to no mode satisfying the condition dmax=0, then selecting a mode for which dmax is below a defined threshold.

### Claim 6

A device for temporal decomposition and reconstruction of an input video signal that uses a prediction process and an update process in a framework of motion compensated temporal filtering (MCTF), a motion information used for the update process being derived from the motion information used for the prediction process, the device employing a block based video codec comprising an encoder and a decoder, and a non-transitory computer-readable storage medium encoded with a computer program executable by a computer and configured to:
receive a residual picture signal having a precision;
receive a low-pass picture signal having a precision that is less than the precision of the residual picture signal;
in response to the residual picture signal being decoded in the decoder, dip the residual picture signal such that the residual picture signal is reduced in precision, such that the low-pass picture signal, a reconstructed picture, and the clipped residual picture signal have equal precisions;
performing an inverse MCTF process, comprising a lifting scheme, on the dipped residual picture signal; and
in response to the block based video codec selecting a block mode for each block, avoiding selecting a block mode in which a residual clipping would lead to breaking a perfect reconstruction property of the MCTF, thereby defining a constrained mode decision,
wherein the constrained mode decision considers a mode in accordance with the following:
define ĥc[x,k]=C(ĥ[x,k]), where
dmax is used to determine whether to select a considered block mode,
x is a vector describing a position of a pixel in a picture,
k describes a temporal position of the picture,
ĥ[x,k] describes the residual picture signal,
C is a clipping function C(w),
ĥC[x,k] are values stored in a storage medium,
C
⁡
(
w
)
=
{
A
min
if
w
<
A
min
,
A
max
if
w
>
A
max
,
w
otherwise
,
Amin and Amax are minimum and maximum values, respectively, that may be represented with b bits of precision in signed form, and w is an attribute of clipping function C(w);
define
⁢
⁢
d
max
=
max
χɛ
⁢
⁢
B
⁢

C
′
⁡
(
h
~
⁡
[
x
,
k
]
)
-
h
~
⁡
[
x
,
k
]

,
where
B is a block being considered,
C′ is di function C modified such that A′min and A′max limits are used instead of Amin and Amax, respectively with A′min≧Amin and A′max≦Amax,
{tilde over (h)}[x,k] is an estimation of the signal ĥ[h,x,k] that will be received at the decoder; and
verify if dmax=0,and if not forbid the considered mode from being selected for block B.

### Claim 7

A device for temporal decomposition and reconstruction of an input video signal that uses a prediction process and an update process in a framework of motion compensated temporal filtering (MCTF), a motion information used for the update process being derived from the motion information used for the prediction process, the device employing a block based video codec comprising an encoder and a decoder, the codec configured to facilitate:
receiving a residual picture signal having a precision;
receiving a low-pass picture signal having a precision that is less than the precision of the residual picture signal;
in response to the residual picture signal being decoded in the decoder, clipping the residual picture signal such that the residual picture signal is reduced in precision,
such that the low-pass picture signal, a reconstructed picture, and the clipped
residual picture signal have equal precisions;
performing an inverse MOT process, comprising a lifting scheme, on the clipped residual picture signal; and
in response to the block based video codec selecting a block mode for each block, avoiding selecting a block mode in which a residual clipping would lead to breaking a perfect reconstruction property of the MCTF, thereby defining a constrained mode decision,
wherein the constrained mode decision considers a mode in accordance with the following:
define ĥC[x,k]=C(ĥ[x,k]), where
dmax is used to determine whether to select a considered block mode,
x is a vector describing a position of a pixel in a picture,
k describes a temporal position of the picture,
ĥ[x,k] describes the residual picture signal,
C is a clipping function C(w),
ĥC[x,k] are values stored in a storage medium,
C
⁡
(
w
)
=
{
A
min
if
w
<
A
min
,
A
max
if
w
>
A
max
,
w
otherwise
,
a Amin and Amax are minimum and maximum vlues, respectively, that may be represented with b bits of precision in signed form, and w is an attribute of function C(w);
define
⁢
⁢
d
max
=
max
χ
⁢
⁢
ɛ
⁢
⁢
B
⁢

C
′
⁡
(
h
~
⁡
[
x
,
k
]
)
-
h
~
⁡
[
x
,
k
]

,
where
B is a block being considered,
C′ is clipping function C modified such that A′min and A′max limits are used instead of Amin and Amax, respectively with A′min≧Amin and A′max≦Amax,
{tilde over (h)}[x,k] is an estimation of the signal ĥ[x,k] that will be received at the decoder; and
verify if dmax=0, and if not, forbid the considered mode from being selected for block B.

### Claim 8

A method for temporal decomposition and reconstruction of an input video signal that uses a prediction process and an update process in framework of motion compensated temporal filtering (MCTF), the motion information used for the update process being derived from the motion information used for the prediction process, the method employing a block based video codec composed of an encoder and a decoder, the method comprising:
receiving a residual picture signal having a precision;
receiving a low-pass picture signal having a precision that is less than the precision of the residual picture signal;
in response to the residual picture signal being decoded in the decoder, clipping the residual picture signal such that the residual picture signal is reduced in precision, such that the low-pass picture signal, a reconstructed picture, and the clipped residual picture signal have equal precisions;
performing an inverse MOT process on the clipped residual picture signal;
in response to the block based video codec selecting a block mode for each block, avoiding selecting a block mode in which a residual clipping would lead to breaking a perfect reconstruction property of the MCTF, thereby defining a constrained mode decision; and
in response to the block based video codec specifying a variation of a prediction process that selects a motion vector for each block, avoiding selecting a motion vector that would lead to breaking the perfect reconstruction property of the MCTF, thereby defining a constrained motion estimation, wherein the constrained mode estimation considers a motion in accordance with the following:
define ĥc[x,k]=C(ĥ[x,k]), where
dmax is used to determine whether to select a considered block mode,
x is a vector describing a position of a pixel in a picture,
k describes a temporal position of the picture,
ĥ[x,k] describes the residual picture signal,
C is a clipping function C(w),
ĥc[x,k] are values stored in a storage medium,
C
⁡
(
w
)
=
{
A
min
if
w
<
A
min
,
A
max
if
w
>
A
max
,
w
otherwise
,
Amin and Amax are minimum and maximum values, respectively, that may be represented with b bits of precision in signed form, and w is an attribute of clipping function C(w);
define
⁢
⁢
d
max
=
max
x
∈
B
⁢

C
′
⁡
(
h
~
⁡
[
x
,
k
]
)
-
h
~
⁡
[
x
,
k
]

,
where
B is a block being considered,
C′ is clipping function C modified such that A′min and A′max limits are used instead of Amin andAmax, respectively with
A′min≧Amin and A′max≦Amax,
{tilde over (h)}[x,k] is an estimation of the signal ĥ[x,k] that will be received at the decoder; and
verify if dmax=0, and if not, forbid the considered mode from being selected for block B.

### Claim 9 (depends on 8)

The method of claim 8, wherein the constrained motion estimation process considers a motion vector in further accordance with the following:
verify if dmax=0, and if not, then forbid the considered motion vector from being selected for block B.

### Claim 10 (depends on 9)

The method of claim 9, wherein:
in response to no motion vector satisfying the condition dmax=0, then selecting a motion vector for which d max is below a defined threshold.
