US 8,804,836 B2Granted

Video coding

AssigneeSkype

InventorsMattias Nilsson, Renat Vafin, Soren Vang Andersen

Life of this patent

Estimating the term…

Loading patent…

Abstract

A method comprising: for each target image portion to be encoded in a frame, selecting one of a set of encoding modes by optimizing a function comprising an estimate of distortion for the target image portion and a measure of bit rate required to encode the target image portion, encoding the target image portion into the encoded video stream using the selected mode. The encoded video stream is transmitted over a lossy channel. An error propagation distortion map is maintained, which comprises a plurality of error propagation distortion values mapping to respective frame partitions, the error propagation distortion values being based on previous encoding mode selections. The estimate of distortion used to select the encoding mode for each of the target image portions is based on a corresponding portion from the error propagation distortion map, and that corresponding portion is constrained to being co-located with the target image portion.

Claims27

Dependent claims

A method of encoding a video stream comprising a plurality of frames, the method comprising:

for each of a plurality of target image portions to be encoded in a frame, selecting a preferred one of a set of encoding modes by optimizing a function comprising an estimate of distortion for the target image portion and a measure of bit rate required to encode the target image portion, and encoding the target image portion into the encoded video stream using the selected encoding mode;
transmitting the encoded video stream over a lossy channel;
and
maintaining an error propagation distortion map which comprises a plurality of error propagation distortion values mapping to respective frame partitions, the error propagation distortion values being based on previous encoding mode selections;
wherein the estimate of distortion used to select the encoding mode for each of the target image portions is based on a corresponding portion from the error propagation distortion map;
wherein said corresponding portion is constrained to being co-located with the target image portion;
and
wherein the method further comprises updating the error propagation distortion map after the encoding mode selection for the target image portions, and using the updated map to determine a subsequent encoding mode selection for each of the target image portions;
and
wherein the update of the error propagation map comprises a lower complexity calculation as a result of said constraint, relative to a case where the corresponding portion is based on a weighted sum of the error propagation distortion values from a plurality of said frame partitions.

7 more elements

A computer program product for encoding a video stream comprising a plurality of frames, the computer program product being embodied on a computer-readable storage memory device and comprising code configured so as when executed to perform the operations comprising:

for each of a plurality of target image portions to be encoded in a frame, selecting a preferred one of a set of encoding modes by optimizing a function comprising an estimate of distortion for the target image portion and a measure of bit rate required to encode the target image portion, and encoding the target image portion into the encoded video stream using the selected encoding mode;
transmitting the encoded video stream over a lossy channel;
and
maintaining an error propagation distortion map which comprises a plurality of error propagation distortion values mapping to respective frame partitions, the error propagation distortion values being based on previous encoding mode selections;
wherein the estimate of distortion used to select the encoding mode for each of the target image portions is based on a corresponding portion from the error propagation distortion map;
wherein said corresponding portion is constrained to being co-located with the target image portion;
and
wherein the code is configured so as when executed to update the error propagation distortion map after the encoding mode selection for the target image portions, and use the updated map to determine a subsequent encoding mode selection for each of the target image portions;
and
wherein the update of the error propagation map comprises a lower complexity calculation as a result of said constraint, relative to a case where the corresponding portion is based on a weighted sum of the error propagation distortion values from a plurality of said frame partitions.

7 more elements

An apparatus for encoding a video stream comprising a plurality of frames, the apparatus comprising:

an encoder arranged, for each of a plurality of target image portions to be encoded in a frame, to select a preferred one of a set of encoding modes by optimizing a function comprising an estimate of distortion for the target image portion and a measure of bit rate required to encode the target image portion, and to encode the target image portion into the encoded video stream using the selected encoding mode;
and
a transmitter for transmitting the encoded video stream over a lossy channel;
and
wherein the encoder is configured to maintain an error propagation distortion map which comprises a plurality of error propagation distortion values mapping to respective frame partitions, the error propagation distortion values being based on previous encoding mode selections, and the estimate of distortion used to select the encoding mode for each of the target image portions being based on a corresponding portion from the error propagation distortion map;
wherein the encoder is configured to constrain said corresponding portion to being co-located with the target image portion;
wherein the encoder is configured to update the error propagation distortion map after the encoding mode selection for the target image portions, and use the updated map to determine a subsequent encoding mode selection for each of the target image portions;
and
wherein the update of the error propagation map comprises a lower complexity calculation as a result of said constraint, relative to a case where the corresponding portion is based on a weighted sum of the error propagation distortion values from a plurality of said frame partitions.

6 more elements

A method of encoding a video stream comprising a plurality of frames, the method comprising:

for each of a plurality of target image portions to be encoded in a frame, selecting a preferred one of a set of encoding modes by optimizing a function comprising an estimate of distortion for the target image portion and a measure of bit rate required to encode the target image portion, and encoding the target image portion into the encoded video stream using the selected encoding mode;
transmitting the encoded video stream over a lossy channel;
and
maintaining an error propagation distortion map which comprises a plurality of error propagation distortion values mapping to respective frame partitions, the error propagation distortion values being based on previous encoding mode selections;
wherein the estimate of distortion used to select the encoding mode for each of the target image portions is based on a corresponding portion from the error propagation distortion map;
wherein said corresponding portion is constrained to being co-located with the target image portion,
wherein the estimate of distortion comprises a first term representing source coding distortion, and a bias term representing an estimate of distortion that would be experienced due to loss over said channel;
wherein the bias term is based on said corresponding portion from the error propagation distortion map,
wherein to maintain the error propagation distortion map, the error propagation distortion value for a frame partition of a current frame is based on a contribution from the error propagation distortion value for that frame partition in a preceding frame and on a contribution representing an estimate of distortion that would be experienced due to concealment in event of loss over said channel,
wherein the concealment term is determined based on an assumption that, in event of loss of a target image portion, the concealment copies the target image portion directly from a co-located portion of a preceding frame.

7 more elements

A method of encoding a video stream comprising a plurality of frames, the method comprising:

for each of a plurality of target image portions to be encoded in a frame, selecting a preferred one of a set of encoding modes by optimizing a function comprising an estimate of distortion for the target image portion and a measure of bit rate required to encode the target image portion, and encoding the target image portion into the encoded video stream using the selected encoding mode;
transmitting the encoded video stream over a lossy channel;
and
maintaining an error propagation distortion map which comprises a plurality of error propagation distortion values mapping to respective frame partitions, the error propagation distortion values being based on previous encoding mode selections;
wherein the estimate of distortion used to select the encoding mode for each of the target image portions is based on a corresponding portion from the error propagation distortion map;
wherein said corresponding portion is constrained to being co-located with the target image portion,
wherein the estimate of distortion comprises a first term representing source coding distortion, and a bias term representing an estimate of distortion that would be experienced due to loss over said channel;
wherein the bias term is based on said corresponding portion from the error propagation distortion map,
wherein:
to maintain the error propagation distortion map, the error propagation distortion value for a frame partition of a current frame is based on a contribution from the error propagation distortion value for that frame partition in a preceding frame and on a contribution representing an estimate of distortion that would be experienced due to concealment in event of loss over said channel;
the encoding modes comprise at least (i) an intra frame mode which encodes the target image portion relative to a reference image portion in the same frame, and (ii) an inter frame encoding mode which encodes the target image portion relative to a reference image portion in a previously encoded frame;
the selected encoding mode oopt is calculated by:
o
opt
=
argmin
o
⁡
(
D
s
⁡
(
m
,
o
)
+
D
ep_ref
⁡
(
m
,
o
)
+
λ
⁢
⁢
R
⁡
(
m
,
o
)
)
,
where Ds(m,o)+Dep—ref(m,o)+λR(m,o) is said function, Ds is the first term, Dep—ref(m, o) is the bias term, R(o) is the bitrate, λ is a factor representing a trade-off between distortion and bitrate, and m is an index of the target image portion;
the method comprises determining a probability p that a packet will be lost over the channel, wherein the concealment term is weighted by a factor of p;
and
the error propagation distortion map is calculated by:
D
ep
⁡
(
m
,
n
+
1
)
=
{
D
ep
⁡
(
m
,
n
)
+
pD
ec
-
rec
⁡
(
m
,
n
,
o
opt
)
if
⁢
⁢
o
opt
=
INTER
pD
ep
⁡
(
m
,
n
)
+
pD
ec
-
rec
⁡
(
m
,
n
,
o
opt
)
if
⁢
⁢
o
opt
=
INTRA
where m is the mth image portion, and Dec-rec is the contribution due to concealment.

121 more elements

Dependencies are inferred from the claim text.

Facts

Application: 13/274,739Filed 17 Oct 2011
Priority: GB 1114323.719 Aug 2011
Published: US 2013/0044804 A121 Feb 2013
Granted: 12 Aug 2014
Term ends: Not estimatedThe source records don’t support an estimate.

Classification7 codes

H04N 19/176
The unit being an image region, e.g. an object: the region being a block, e.g. a macroblock
H04N 19/103
Characterised by the element, parameter or selection affected or controlled by the adaptive coding: selection of coding mode or of prediction mode
H04N 19/107
Selection of coding mode or of prediction mode: between spatial and temporal predictive coding, e.g. picture refresh
H04N 19/166
Feedback from the receiver or from the transmission channel: concerning the amount of transmission errors, e.g. bit error rate [BER]