# US 2012/0084453 A1 · Adjusting audio and video synchronization of 3g tdm streams

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2012/0084453 A1
- Document ID: US20120084453A1
- Status: Published application
- Kind: A1
- Application number: 12/897,050
- Filed: 2010-10-04
- Published: 2012-04-05
- Earliest claimed: 2010-10-04
- Assignees: Dialogic Corporation
- Inventors: Mark L. Buser; Ranjan Singh
- Web page: https://patentgrep.com/patent/US20120084453A1

## CPC

- H04N 21/2368: Multiplexing of audio and video streams
- H04N 21/242: Synchronisation processes, e.g. processing of PCR [Programme Clock References] {(arrangements for synchronising broadcast or distribution via plural systems in broadcast distribution systems H04H20/18)}
- H04N 21/44209: {Monitoring of downstream path of the transmission network originating from a server, e.g. bandwidth variations of a wireless network (arrangements for maintenance or administration in data switching networks involving bandwidth and capacity management H04L41/0896)}
- H04N 21/6131: {involving transmission via a mobile phone network (wireless downlink channel access H04W74/006)}
- H04N 21/6437: Real-time Transport Protocol [RTP]
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Systems and methods of adjusting synchronization of audio media streams and video media streams in 3G mobile communications systems that can mitigate the effects of temporal skew due to intervening processing elements associated with media channels carrying the respective media streams. The systems and methods are operative to adjust the synchronization of audio media streams and video media streams by receiving control messages that report delays due to such intervening processing elements, calculating a relative amount of delay using the reported delays for each media channel, and applying a delay factor based on the relative amount of delay to the faster media channel to place the audio media streams and the video media streams in proper temporal alignment. The delay factor is applied to the faster media channel at those locations within the mobile communications systems where the audio and video media streams are combined and/or separated for subsequent transmission.

## Claims

### Claim 1

A method of adjusting synchronization of an audio media stream and a video media stream in a mobile communications system, the audio media stream and the video media stream being carried by an audio media channel and a video media channel, respectively, at least one of the audio media channel and the video media channel having at least one media processing component associated therewith for use in processing the audio media stream and the video media stream carried by the audio media channel and the video media channel, respectively, thereby subjecting at least one of the audio media stream and the video media stream to at least one delay, the method comprising the steps of:
receiving, by at least one processor, at least one message reporting the at least one delay due to the at least one media processing component associated with the audio media channel and the video media channel;
calculating, by the at least one processor, a relative amount of delay between the audio media channel and the video media channel using the at least one delay reported by the at least one message;
determining, by the at least one processor, a faster one of the audio media channel and the video media channel, such determination being based at least in part on the at least one delay reported by the at least one message; and
applying, at a media endpoint within the mobile communications system, a delay factor based on the relative amount of delay to the faster one of the audio media channel and the video media channel, thereby adjusting the synchronization of the audio media stream and the video media stream.

### Claim 2 (depends on 1)

The method of claim 1 further comprising:
reading, by the at least one processor, at least one packet from the respective media stream carried by the faster one of the audio media channel and the video media channel, and determining, by the at least one processor, at least one arrival time of the at least one packet.

### Claim 3 (depends on 2)

The method of claim 2 wherein the applying of the delay factor includes applying the delay factor to the faster one of the audio media channel and the video media channel in accordance with at least the at least one arrival time of the at least one packet.

### Claim 4 (depends on 1)

The method of claim 1 further comprising:
reading, by the at least one processor, at least one packet from the respective media stream carried by the faster one of the audio media channel and the video media channel, and obtaining at least one payload from the at least one packet.

### Claim 5 (depends on 4)

The method of claim 4 further comprising:
storing the at least one payload in a queue at the media endpoint.

### Claim 6 (depends on 5)

The method of claim 5 wherein the queue includes a plurality of buffers, and wherein the storing of the at least one payload includes storing the at least one payload in at least one of the plurality of buffers.

### Claim 7 (depends on 6)

The method of claim 6 wherein the applying of the delay factor includes appending the delay factor to the at least one of the plurality of buffers in which the at least one payload is stored.

### Claim 8 (depends on 7)

The method of claim 7 wherein the applying of the delay factor includes aging the delay factor.

### Claim 9 (depends on 8)

The method of claim 8 further comprising:
responsive to the delay factor being aged at least approximately to zero, reading, at the media endpoint, the at least one payload from the queue.

### Claim 10 (depends on 9)

The method of claim 9 further comprising:
assembling, at the media endpoint, at least one multimedia packet containing the at least one payload.

### Claim 11 (depends on 10)

The method of claim 10 further comprising:
forming at least one frame from the at least one multimedia packet.

### Claim 12 (depends on 10)

The method of claim 10 further comprising:
encapsulating the at least one multimedia packet.

### Claim 13 (depends on 10)

The method of claim 10 further comprising:
transmitting, from the media endpoint, the at least one multimedia packet over at least one network.

### Claim 14 (depends on 13)

The method of claim 13 wherein the at least one network includes an Internet protocol (IP) network, and wherein the transmitting of the at least one multimedia packet includes transmitting, from the media endpoint, the at least one multimedia packet over the IP network.

### Claim 15 (depends on 13)

The method of claim 13 wherein the at least one network includes a circuit-switched wireless network, and wherein the transmitting of the at least one multimedia packet includes transmitting, from the media endpoint, the at least one multimedia packet over the circuit-switched wireless network.

### Claim 16 (depends on 1)

The method of claim 1 wherein the at least one processor is included in the media endpoint, and wherein the receiving of the at least one message includes receiving the at least one message by the at least one processor within the media endpoint.

### Claim 17 (depends on 1)

The method of claim 1 wherein the at least one processor is included in a central controller within the mobile communications system, and wherein the receiving of the at least one message includes receiving the at least one message by the at least one processor within the central controller.

### Claim 18 (depends on 17)

The method of claim 17 further comprising:
transmitting, from the central controller, at least one report of one or more of the at least one delay due to the at least one media processing component and the relative amount of delay to the media endpoint.

### Claim 19 (depends on 17)

The method of claim 17 wherein one or more of the calculating of the relative amount of delay and the determining of the faster one of the audio media channel and the video media channel are distributed among the media endpoint and the central controller.

### Claim 20

A system for adjusting synchronization of an audio media stream and a video media stream, the audio media stream and the video media stream being carried by an audio media channel and a video media channel, respectively, at least one of the audio media channel and the video media channel having at least one media processing component associated therewith for use in processing the audio media stream and the video media stream carried by the audio media channel and the video media channel, respectively, thereby subjecting at least one of the audio media stream and the video media stream to at least one delay, the system comprising:
at least one media endpoint operative:
to receive at least one message reporting the at least one delay due to the at least one media processing component associated with the audio media channel and the video media channel;
to calculate a relative amount of delay between the audio media channel and the video media channel using the at least one delay reported by the at least one message;
to determine a faster one of the audio media channel and the video media channel based at least in part on the at least one delay reported by the at least one message; and
to apply a delay factor based on the relative amount of delay to the faster one of the audio media channel and the video media channel, thereby adjusting the synchronization of the audio media stream and the video media stream.

### Claim 21 (depends on 20)

The system of claim 20 wherein the at least one media endpoint is further operative to read at least one packet from the respective media stream carried by the faster one of the audio media channel and the video media channel, and to determine at least one arrival time of the at least one packet.

### Claim 22 (depends on 21)

The system of claim 21 wherein the at least one media endpoint is further operative to apply the delay factor to the faster one of the audio media channel and the video media channel according to the at least one arrival time of the at least one packet.

### Claim 23 (depends on 20)

The system of claim 20 wherein the at least one media endpoint is further operative to read at least one packet from the respective media stream carried by the faster one of the audio media channel and the video media channel, and to obtain at least one payload from the at least one packet.

### Claim 24 (depends on 23)

The system of claim 23 wherein the at least one media endpoint includes at least one queue, and wherein the at least one media endpoint is further operative to store the at least one payload in the at least one queue.

### Claim 25 (depends on 24)

The system of claim 24 wherein the queue includes a plurality of buffers, and wherein the at least one media endpoint is further operative to store the at least one payload in at least one of the plurality of buffers.

### Claim 26 (depends on 25)

The system of claim 25 wherein the at least one media endpoint is further operative to append the delay factor to the at least one of the plurality of buffers in which the at least one payload is stored.

### Claim 27 (depends on 26)

The system of claim 26 wherein the at least one media endpoint is further operative to age the delay factor.

### Claim 28 (depends on 27)

The system of claim 27 wherein the at least one media endpoint is further operative, in response to the delay factor being aged at least approximately to zero, to read the at least one payload from the queue.

### Claim 29 (depends on 28)

The system of claim 28 wherein the at least one media endpoint is further operative to assemble at least one multimedia packet containing the at least one payload.

### Claim 30 (depends on 29)

The system of claim 29 wherein the at least one media endpoint is further operative to form at least one frame from the at least one multimedia packet.

### Claim 31 (depends on 29)

The system of claim 29 wherein the at least one media endpoint is further operative to encapsulate the at least one multimedia packet.

### Claim 32 (depends on 29)

The system of claim 29 wherein the at least one media endpoint is further operative to transmit the at least one multimedia packet over at least one network.

### Claim 33 (depends on 32)

The system of claim 32 wherein the at least one network includes an Internet protocol (IP) network, and wherein the at least one media endpoint is further operative to transmit the at least one multimedia packet over the IP network.

### Claim 34 (depends on 33)

The system of claim 33 wherein the at least one media endpoint includes a multiplexer media endpoint.

### Claim 35 (depends on 32)

The system of claim 32 wherein the at least one network includes a circuit-switched wireless network, and wherein the at least one media endpoint is further operative to transmit the at least one multimedia packet over the circuit-switched wireless network.

### Claim 36 (depends on 35)

The system of claim 35 wherein the at least one media endpoint includes a de-multiplexer media endpoint.

### Claim 37

A system for adjusting synchronization of an audio media stream and a video media stream, the audio media stream and the video media stream being carried by an audio media channel and a video media channel, respectively, at least one of the audio media channel and the video media channel having at least one media processing component associated therewith for use in processing the audio media stream and the video media stream carried by the audio media channel and the video media channel, respectively, thereby subjecting at least one of the audio media stream and the video media stream to at least one delay, the system comprising:
one or more processors operative to execute a plurality of functions: to receive at least one control message reporting the at least one delay due to the at least one media processing component associated with the audio media channel and the video media channel;
to calculate a relative amount of delay between the audio media channel and the video media channel using the at least one delay reported by the at least one control message;
to determine a faster one of the audio media channel and the video media channel based at least in part on the at least one delay reported by the at least one control message; and
to apply a delay factor based on the relative amount of delay to the faster one of the audio media channel and the video media channel, thereby adjusting the synchronization of the audio media stream and the video media stream.

### Claim 38 (depends on 37)

The system of claim 37 further comprising a media endpoint, and wherein the one or more processors are included in the media endpoint.

### Claim 39 (depends on 37)

The system of claim 37 further comprising a central controller, and wherein the one or more processors are included in the central controller.

### Claim 40 (depends on 37)

The system of claim 37 further comprising a media endpoint and a central controller, and wherein a portion of the one or more processors is included in each of the media endpoint and the central controller.

### Claim 41 (depends on 40)

The system of claim 40 wherein the plurality of functions executed by the one or more processors are distributed among the media endpoint and the central controller.
