US 9,756,356 B2Granted

Application-assisted spatio-temporal error concealment for RTP video

AssigneeDialogic Corporation

InventorsKyeong Ho Yang, Myo Tun, Ashwini Kumar

Life of this patent

Estimating the term…

Loading patent…

Abstract

Systems and methods that employ an application-assisted approach to adaptive spatio-temporal error concealment in video decoders. The systems and methods employ a video receiver that can receive real-time video frames from a video transmitter over a wired and/or wireless network. The video receiver includes an end system configurable as an application for consuming the content of the real-time video frames. The end system application can determine whether each received real-time video frame is complete or incomplete, whether each received slice is complete or incomplete, and provide such information to the video decoder for use in avoiding errors while decoding the respective real-time video frames. The video decoder can determine whether any pixel blocks included in real-time video packets are properly decoded or missing, generate statistics regarding the coding mode of the respective pixel blocks, and use such further information while performing adaptive spatio-temporal error concealment on the respective pixel blocks.

Claims11

Dependent claims

A method of application-assisted error concealment in a video receiver, the video receiver including an end system application and a video decoder, the video decoder having an application-assisted error concealment component, comprising:

receiving, at the video receiver over a packet communications network, a plurality of real-time video frames, each of the plurality of real-time video frames including a plurality of slices, each of the plurality of slices including a plurality of real-time video packets, and each of the plurality of real-time video packets including a plurality of pixel blocks;
obtaining, by the end system application, first predetermined information as to (1) whether each respective real-time video frame in the plurality of real-time video frames is complete or incomplete, (2) whether each respective slice in the plurality of slices in the respective real-time video frame is complete, incomplete, or missing, (3) whether at least one of the plurality of real-time video packets in the respective slice is missing, and (4) in the event at least one of the plurality of real-time video packets is missing, whether at least one of the plurality of real-time video packets remaining in the respective slice requires information from the at least one missing real-time video packet for proper decoding;
decoding, by the video decoder, the plurality of real-time video frames including the plurality of pixel blocks included in the plurality of real-time video packets using at least the first predetermined information;
determining, by the video decoder, whether the plurality of pixel blocks are properly decoded or improperly decoded;
obtaining, by the application-assisted error concealment component, a list of pixel blocks eligible for error concealment, the list of pixel blocks including indications of one or more improperly decoded or missing pixel blocks;
and
for each respective pixel block in the list of pixel blocks:
obtaining, by the application-assisted error concealment component, second predetermined information pertaining to one or more neighboring pixel blocks, the second predetermined information including indications of (1) an availability of the respective neighboring pixel blocks, (2) in the event at least one neighboring pixel block is available, a coding mode of the at least one neighboring pixel block, and (3) in the event the at least one neighboring pixel block has already undergone error concealment, a mode of error concealment employed on the at least one neighboring pixel block;
evaluating, by the application-assisted error concealment component using at least the second predetermined information, a plurality of modes of error concealment for possible use on the respective pixel block to obtain an evaluation result;
and
performing, by the application-assisted error concealment component, one of the plurality of modes of error concealment on the respective pixel block based on the evaluation result.

8 more elements

A video receiver for use in receiving, over a packet communications network, a plurality of real-time video frames, each of the plurality of real-time video frames including a plurality of slices, each of the plurality of slices including a plurality of real-time video packets, and each of the plurality of real-time video packets including a plurality of pixel blocks, the video receiver comprising:

an end system application operative to obtain first predetermined information as to (1) whether each respective real-time video frame in the plurality of real-time video frames is complete or incomplete, (2) whether each respective slice in the plurality of slices in the respective real-time video frame is complete, incomplete, or missing, (3) whether at least one of the plurality of real-time video packets in the respective slice is missing, and (4) in the event at least one of the plurality of real-time video packets is missing, whether at least one of the real-time video packets remaining in the respective slice requires information from the at least one missing real-time video packet for proper decoding;
a video decoder operative:
to decode the plurality of real-time video frames including the plurality of pixel blocks included in the plurality of real-time video packets using at least the first predetermined information;
and
to determine whether the plurality of pixel blocks are properly decoded or improperly decoded;
and
an application-assisted error concealment component operative:
to obtain a list of pixel blocks eligible for error concealment, the list of pixel blocks including indications of one or more improperly decoded or missing pixel blocks;
and
for each respective pixel block in the list of pixel blocks:
to obtain second predetermined information pertaining to one or more neighboring pixel blocks, the second predetermined information including indications of (1) an availability of the respective neighboring pixel blocks, (2) in the event at least one neighboring pixel block is available, a coding mode of the at least one neighboring pixel block, and (3) in the event the at least one neighboring pixel block has already undergone error concealment, a mode of error concealment employed on the at least one neighboring pixel block;
using at least the second predetermined information, to evaluate a plurality of modes of error concealment for possible use on the respective pixel block to obtain an evaluation result;
and
to perform one of the plurality of modes of error concealment on the respective pixel block based on the evaluation result.

11 more elements

A system for application-assisted error concealment in real-time communications, the system comprising:

a video transmitter operative to send a plurality of real-time video frames over a packet communications network, each of the plurality of real-time video frames including a plurality of slices, each of the plurality of slices including a plurality of real-time video packets, and each of the plurality of real-time video packets including a plurality of pixel blocks;
and
a video receiver operative to receive the plurality of real-time video frames sent over the packet communications network, the video receiver including:
an end system application operative to obtain first predetermined information as to (1) whether each respective real-time video frame in the plurality of real-time video frames is complete or incomplete, (2) whether each respective slice in the plurality of slices in the respective real-time video frame is complete, incomplete, or missing, (3) whether at least one of the plurality of real-time video packets in the respective slice is missing, and (4) in the event at least one of the plurality of real-time video packets is missing, whether at least one of the real-time video packets remaining in the respective slice requires information from the at least one missing real-time video packet for proper decoding;
a video decoder operative:
to decode the plurality of real-time video frames including the plurality of pixel blocks included in the plurality of real-time video packets using at least the first predetermined information;
and
to determine whether the plurality of pixel blocks are properly decoded or improperly decoded;
and
an application-assisted error concealment component operative:
to obtain a list of pixel blocks eligible for error concealment, the list of pixel blocks including indications of one or more improperly decoded or missing pixel blocks;
and
for each respective pixel block in the list of pixel blocks:
to obtain second predetermined information pertaining to one or more neighboring pixel blocks, the second predetermined information including indications of (1) an availability of the respective neighboring pixel blocks, (2) in the event at least one neighboring pixel block is available, a coding mode of the at least one neighboring pixel block, and (3) in the event the at least one neighboring pixel block has already undergone error concealment, a mode of error concealment employed on the at least one neighboring pixel block;
using at least the second predetermined information, to evaluate a plurality of modes of error concealment for possible use on the respective pixel block to obtain an evaluation result;
and
to perform one of the plurality of modes of error concealment on the respective pixel block based on the evaluation result.

14 more elements

Dependencies are inferred from the claim text.

Facts

Application: 14/307,860Filed 18 Jun 2014 · us provisional application
Priority: 24 Jun 2013
Published: US 2014/0376632 A125 Dec 2014
Granted: 5 Sep 2017
Term ends: Not estimatedThe source records don’t support an estimate.

Classification3 codes

H04N 19/521
Processing of motion vectors: for estimating the reliability of the determined motion vectors or motion vector field, e.g. for smoothing the…
H04N 19/895
Involving methods or arrangements for detection of transmission errors at the decoder: in combination with error concealment
H04N
Pictorial communication