US 9,521,057 B2Granted

Adaptive audio stream with latency compensation

AssigneeAmazon Technologies, Inc.

InventorsMichael Klingbeil, Adam Marks

Life of this patent

Estimating the term…

Loading patent…

Abstract

A latency compensating adaptive jitter buffer method is implemented in an audio client running on a user device or in an audio server to adaptively adjust the size of a jitter buffer to optimize latency while minimizing packet loss during audio signal transmission. In some embodiments, the jitter buffer is kept to a nominal size for low latency. In response to a queuing delay event being detected, audio production is temporarily stopped and the size of the jitter buffer is temporarily increased to receive all incoming audio packets up to a certain refill level. The method then resumes audio production using accelerated playback to reduce the jitter buffer size back to the nominal size.

Claims20

Dependent claims

A method in a receive process of an audio client being executed on a user device or an audio conferencing server for receiving audio data packets, the method comprising:

setting a jitter buffer to a nominal size for low latency;
receiving incoming audio data packets;
storing the audio data packets into the jitter buffer;
producing audio playback using the audio data packets from the jitter buffer at a nominal playback rate;
monitoring the incoming audio data packets for a queuing delay event;
in response to a queuing delay event being detected, stopping production of audio playback;
increasing the size of the jitter buffer to greater than the nominal size to receive and store incoming audio data packets;
storing audio data packets in the jitter buffer up to a refill level greater than the nominal size;
in response to the jitter buffer having stored therein audio data packets reaching the refill level, resuming production of audio playback using the audio data packets from the jitter buffer at an accelerated playback rate faster than the nominal playback rate;
and
in response to the jitter buffer size returning to the nominal size, continuing to receive incoming audio data packets and producing audio playback of the audio data packets from the jitter buffer at the nominal playback rate.

8 more elements

A cloud-based audio conferencing system for processing audio signals of a conference call, comprising:

a processor configured to execute a receive process for receiving audio data packets, the receive process being configured to:
set a jitter buffer to a nominal size for low latency;
receive incoming audio data packets;
store the audio data packets into the jitter buffer;
produce audio playback using the audio data packets from the jitter buffer at a nominal playback rate;
monitor the incoming audio data packets for a queuing delay event;
in response to a queuing delay event being detected, stop production of audio playback;
increase the size of the jitter buffer to greater than the nominal size to receive and store incoming audio data packets;
store audio data packets in the jitter buffer up to a refill level greater than the nominal size;
in response to the jitter buffer having stored therein audio data packets reaching the refill level, resume production of audio playback using the audio data packets from the jitter buffer at an accelerated playback rate faster than the nominal playback rate;
and
in response to the jitter buffer size returning to the nominal size, continue to receive incoming audio data packets and producing audio playback of the audio data packets from the jitter buffer at the nominal playback rate;
and
a memory coupled to the processor and configured to provide the processor with instructions.

11 more elements

A device for processing audio signals in an audio communication over a data network, comprising:

a processor configured to execute a receive process for receiving audio data packets, the receive process being configured to:
set a jitter buffer to a nominal size for low latency;
receive incoming audio data packets;
store the audio data packets into the jitter buffer;
produce audio playback using the audio data packets from the jitter buffer at a nominal playback rate;
monitor the incoming audio data packets for a queuing delay event;
in response to a queuing delay event being detected, stop production of audio playback;
increase the size of the jitter buffer to greater than the nominal size to receive and store incoming audio data packets;
store audio data packets in the jitter buffer up to a refill level greater than the nominal size;
in response to the jitter buffer having stored therein audio data packets reaching the refill level, resume production of audio playback using the audio data packets from the jitter buffer at an accelerated playback rate faster than the nominal playback rate;
and
in response to the jitter buffer size returning to the nominal size, continue to receive incoming audio data packets and producing audio playback of the audio data packets from the jitter buffer at the nominal playback rate;
and
a memory coupled to the processor and configured to provide the processor with instructions.

11 more elements

Dependencies are inferred from the claim text.

Facts

Application: 14/581,321Filed 23 Dec 2014 · us provisional application
Priority: 14 Oct 2014
Published: US 2016/0105473 A114 Apr 2016
Granted: 13 Dec 2016
Term ends: Not estimatedThe source records don’t support an estimate.

Classification11 codes

H04L 43/087
Delays: jitter
H04L 47/2416
Traffic characterised by specific attributes, e.g. priority or QoS: real-time traffic
H04L 47/283
In relation to timing considerations: in response to processing delays, e.g. caused by jitter or round trip time [RTT]
H04L 47/30
Flow control; Congestion control: in combination with information about buffer occupancy at either end or at transit nodes

Family

Application62/063,868Us provisional application · filed 14 Oct 2014
US 2016/0105473 A1Pre-grant publication · published 14 Apr 2016
US 9,521,057 B2This patent

From the US filing history. Foreign family members aren’t listed.

Timeline

Citations

Open elsewhere

Building an agent or pulling many patents? Use the API or MCP server instead of scraping pages. Get an API key