# US 7,466,755 B2 · Method for video error concealment by updating statistics

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 7,466,755 B2
- Document ID: US7466755B2
- Status: Granted
- Kind: B2
- Application number: 10/410,274
- Filed: 2003-04-10
- Published: 2004-04-08
- Granted: 2008-12-16
- Earliest claimed: 2002-10-04
- Assignees: Industrial Technology Research Institute
- Inventors: Tsuhan Chen; Trista Pei-Chun Chen; Yuh-Feng Hsu
- Web page: https://patentgrep.com/patent/US7466755B2

## CPC

- H04N 19/895: in combination with error concealment
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A method of error concealment that includes providing a first set of image data at a first time instant, providing a second set of image data at a second time instant, representing the first set of image data by a model of updating mixture of principal components (UMPC) to form a UMPC model having a plurality of components, receiving the second set of image data, wherein the second set of image data contains error-free data, and updating the UMPC model using the error-free data at the second time instant.

## Claims

### Claim 1

A method of error concealment, comprising:
providing a first set of image data at a first time instant;
providing a second set of image data at a second time instant;
representing the first set of image data by a model of updating a mixture of principal components (UMPC) to form a UMPC model having a plurality of components;
receiving the second set of image data, wherein the second set of image data contains error-free data; and
updating the UMPC model using the error-free data at the second time instant,
wherein minimizing the error of representation of the error-free data at the second time instant by the UMPC model is achieved through iteration of the following:
min
m
j
(
n
)
,
U
j
(
n
)
,
w
i
∀
i
,
j
⁢
(
1
∑
i
=
0
∞
⁢
α
i
)
⁢
∑
i
=
0
∞
⁢
α
i
⁢

x
n
-
i
-
∑
j
=
1
M
⁢
w
n
-
i
,
j
[
m
j
(
n
)
+
∑
k
=
1
P
⁢
[
(
x
n
-
i
-
m
j
(
n
)
)
T
⁢
u
jk
(
n
)
]
⁢
u
jk
(
n
)
︸
x
^
n
-
i
,
j
]

2
wherein
n and i are time;
M is a number of components in the UMPC;
xi is a data vector at time i;
mjn is a mean of the jth mixture component estimated at time n;
ujk(n) is a kth eigen-vector of the jth mixture component estimated at time n;
Uj(n) is a matrix with P columns of ujk(n) , wherein k=1˜P;
{circumflex over (x)}ij is a reconstruction of xi with mixture component j;
{circumflex over (x)}i is a matrix with M columns of {circumflex over (x)}ij, j=1˜M;
w ij is a weight of {circumflex over (x)}^ij to reconstruct xi;
wi is a vector with M entries of wij;
αis a decay factor, wherein 0<α<1; and
q, r are indices of the mixture component.

### Claim 2 (depends on 1)

The method of claim 1 wherein an initial value of a mean mq(n) of mixture of component q at time n is determined by:
m
q
(
n
)
=
(
1
-
w
nq
2
∑
i
=
0
∞
⁢
α
i
⁢
w
n
-
i
,
q
2
)
⁢
m
q
(
n
-
1
)
+
(
w
nq
∑
i
=
0
∞
⁢
α
i
⁢
w
n
-
i
,
q
2
)
⁢
(
x
n
-
∑
j
=
1
,
j
≠
q
M
⁢
w
nj
⁢
x
^
nj
)
.

### Claim 3 (depends on 1)

The method of claim 1 wherein an initial value of a covariance matrix Cr(n) of mixture component r at time n is determined by:
C
r
(
n
)
=
⁢
α
⁢
⁢
C
r
(
n
-
1
)
+
(
1
-
α
)
⁢
{
[
(
x
n
-
w
nr
⁢
m
r
(
n
)
)
-
∑
j
=
1
,
j
≠
r
M
⁢
w
nj
⁢
x
^
nj
]
⁡
[
w
nr
⁡
(
x
n
-
m
r
(
n
)
)
]
T
+
[
w
nr
⁡
(
x
n
-
m
r
(
n
)
)
]
⁡
[
(
x
n
-
w
nr
⁢
m
r
(
n
)
)
-
∑
j
=
1
,
j
≠
r
M
⁢
w
nj
⁢
x
^
nj
]
T
-
w
nr
2
⁡
(
x
n
-
m
r
(
n
)
)
⁢
(
x
n
-
m
r
(
n
)
)
T
}
.

### Claim 4 (depends on 1)

The method of claim 1 wherein initial values of the weights are determined by:
[
w
n
-
i
λ
]
=
[
2
⁢
X
^
n
-
i
T
⁢
X
^
n
-
i
1
1
T
0
]
-
1
⁡
[
2
⁢
X
^
n
-
i
T
⁢
x
n
-
i
1
]
wherein 1=[1 . . . 1 ]t is an M×1 vector.

### Claim 5

A method of error concealment, comprising:
representing a set of image data by a model of updating mixture of principal components (UMPC), each principal component of the UMPC model having a mean, a set of eigenvectors, and a weight at any given time instant, wherein the UMPC model is used for concealing video errors;
providing the set of image data at a first time instant and at any time instant before the first time instant;
receiving the set of image data at a second time instant, wherein the set of image data at the second time instant contains at least one error-free region of interest (ROI);
updating the UMPC model to minimize a representation error of the at least one error-free ROI at the second time instant by the UMPC model;
receiving the set of image data at a third time instant, wherein the set of image data at the third time instant contains at least one corrupted ROI; and
reconstructing the at least one corrupted ROI at the third time instant using the updated UMPC model at the second time instant, comprising:
reconstructing the at least one corrupted ROI using each component of the updated UMPC model; and
combining the reconstructed ROI from each component for the at least one corrupted ROI;
wherein a combination coefficient of the reconstructed ROI from each component is inversely proportional to a reconstruction error of the representation of the corrupted ROI with the each component.

### Claim 6 (depends on 5)

The method of claim 5, wherein the model of the UMPC is updated with the at least one error-free ROI when the representation error of the at least one error-free ROI by the UMPC model exceeds a threshold.

### Claim 7 (depends on 5)

The method of claim 5, wherein the set of image data at the second time instant contains a plurality of error-free ROIs, and all of the plurality of error-free ROIs are used to update the UMPC model.

### Claim 8 (depends on 5)

The method of claim 5, wherein the second time instant is the same as the third time instant.

### Claim 9 (depends on 5)

The method of claim 5, wherein the UMPC is updated in non-contiguous time instants.

### Claim 10 (depends on 5)

The method of claim 5, wherein updating the UMPC model comprises
initializing the mean, the set of eigenvectors, and the weight of each component of the UMPC at the second time instant;
for each component of the UMPC, solving for the mean, the set of eigenvectors, and the weight at the second time instant by iterating the following steps until said means, said set of eigenvectors, and said weight converge or a predefined condition such as the number of iterations is achieved:
optimizing and updating the mean with the set of eigenvectors and the weights fixed,
optimizing and updating the set of eigenvectors with the mean and the weight fixed, and
optimizing and updating the weight with the mean and the set of eigenvectors fixed.

### Claim 11 (depends on 5)

The method of claim 5, wherein the combination of the reconstructed ROI from each component is a linear combination.

### Claim 12 (depends on 5)

The method of claim 5, further comprising
replacing the corrupted ROI with reconstructed ROI, and
repeating the step of reconstructing the at least one corrupted ROI and replacing the corrupted ROI with the reconstructed ROI until the reconstructed ROI is satisfactory.