# US 2022/0013131 A1 · Method, a system, and a program for playing back audio/video signals with automatic adjustment of latency

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0013131 A1
- Document ID: US20220013131A1
- Status: Published application
- Kind: A1
- Application number: 17/372,164
- Filed: 2021-07-09
- Published: 2022-01-13
- Earliest claimed: 2020-07-10
- Assignees: Sagemcom Broadband SAS
- Applicants: Sagemcom Broadband SAS
- Inventors: Gabriel BOUVIGNE; Vincent SCHOTT; Frédéric SODI
- Web page: https://patentgrep.com/patent/US20220013131A1

## CPC

- G10L 19/167: {Audio streaming, i.e. formatting and decoding of an encoded audio signal representation into a data stream for transmission or storage purposes}
- H04N 21/44004: {involving video buffer management, e.g. video decoder buffer or video display buffer}
- G10L 19/005: Correction of errors induced by the transmission channel, if related to the coding algorithm
- H04N 21/4302: {Content synchronisation processes, e.g. decoder synchronisation}
- H04N 21/4305: {Synchronising client clock from received content stream, e.g. locking decoder clock with encoder clock, extraction of the PCR packets (arrangements for synchronising receiver with transmitter by comparing receiver clock with transmitter clock H04L7/0012; arrangements for synchronising receiver with transmitter wherein the receiver takes measures against momentary loss of synchronisation H04L7/0083)}
- H04N 21/4307: {Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen}
- H04N 21/4341: {Demultiplexing of audio and video streams}
- H04N 21/43615: {Interfacing a Home Network, e.g. for connecting the client to a plurality of peripherals (home Audio Video Interoperability [HAVI] data switching networks H04L12/2805)}
- H04N 21/4392: {involving audio buffer management}
- H04N 21/8456: {by decomposing the content in the time domain, e.g. in time segments}
- H04N 21/8547: involving timestamps for synchronizing content
- H04R 5/04: Circuit arrangements, {e.g. for selective connection of amplifier inputs/outputs to loudspeakers, for loudspeaker detection, or for adaptation of settings to personal preferences or hearing impairments (combinations of amplifiers H03F3/68; stereophonic systems H04S)}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K)

## Abstract

A method of playing back audio/video signals by means of an audio/video system includes a decoder appliance connected to a display screen and to a wireless speaker including a buffer storing audio frames so as to allow latency between receiving encoded audio frames and supplying decoded audio frames. The method comprises the steps of: determining a filling level of the buffer; comparing the filling level of the buffer with a filling target for the buffer and deducing a filling difference therefrom; and increasing or decreasing the number of audio frames that are stored as a function of the filling difference so as to adjust latency.

## Claims

### Claim 1

A method of playing back audio/video signals by means of an audio/video system comprising a decoder appliance connected to a display screen and to at least one wireless speaker, the decoder appliance comprising a demultiplexer having an input connected to a source of audio/video data, a first output connected to a video processor circuit arranged to supply video data to the display screen, and a second output connected to an audio processor circuit arranged to supply encoded audio frames to a radio transmitter, the speaker including a radio receiver connected to a decoder circuit arranged to supply a sound playback member with decoded audio frames resulting from decoding the encoded audio frames, the speaker including a clock that is synchronized with a clock of the decoder and a buffer for storing audio frames so as to allow latency between receiving encoded audio frames and supplying decoded audio frames, the method comprising the steps of:
determining a filling level of the buffer of said at least one speaker;
comparing the filling level of the buffer of said at least one speaker with a filling target and deducing a filling difference therefrom;
increasing or decreasing the number of audio frames that are stored as a function of the filling difference so as to adjust the latency.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the number of stored audio frames is increased by retarding the clocks by a predetermined duration and the number of stored audio frames is decreased by advancing the clocks by a predetermined duration.

### Claim 3 (depends on 2)

The method according to claim 2, wherein the predetermined duration is less that are equal to about 150 μs.

### Claim 4 (depends on 1)

The method according to claim 1, wherein a presentation timestamp is associated with each image of video data and with each corresponding audio frame, with the number of stored audio frames being increased by adding a predetermined duration to the presentation timestamps, and with the number of stored audio frames being decreased by subtracting a predetermined duration from the presentation timestamps.

### Claim 5 (depends on 4)

The method according to claim 4, wherein the predetermined duration is less than or equal to about 150 μs.

### Claim 6 (depends on 2)

The method according to claim 2, including a step, after decreasing latency, of:
eliminating samples from an audio frame before transmission to the sound playback member or re-sampling an audio frame;
optionally, eliminating an image from the video data.

### Claim 7 (depends on 2)

The method according to claim 2, including a step, after increasing latency, of:
adding samples to an audio frame before transmission to the sound playback member or re-sampling an audio frame or interpolating samples in an audio frame;
optionally, inserting an image into the video data, e.g. by duplication or by interpolation.

### Claim 8 (depends on 1)

The method according to claim 1, wherein the number of stored audio frames is decreased if the filling difference is greater than or equal to a predetermined percentage above the filling target.

### Claim 9 (depends on 1)

The method according to claim 1, wherein the filling target is determined from past filling levels of the buffer.

### Claim 10 (depends on 9)

The method according to claim 9, wherein the filling target is equal to the sum of a safety value plus a margin.

### Claim 11 (depends on 10)

The method according to claim 10, wherein the margin is equal to the minimum filling level over a predetermined past duration.

### Claim 12 (depends on 10)

The method according to claim 10, wherein the margin is equal to one-tenth of the sum of the preceding margin plus nine times the present filling level.

### Claim 13 (depends on 10)

The method according to claim 10, wherein the safety value corresponds to a filling level allowing latency in the range 70 ms to 200 ms.

### Claim 14 (depends on 12)

The method according to claim 12, wherein the safety level is increased if the margin has been negative during a predetermined time lapse for a number of times that is greater than a threshold.

### Claim 15 (depends on 10)

The method according to claim 10, comprising a first stage of adjusting filling strongly when the filling level is less than a first threshold value corresponding to a first percentage of the safety value.

### Claim 16 (depends on 15)

The method according to claim 15, wherein the first threshold value is equal to about 80% of the safety value, and the latency is modified by a value less than or equal to 20 ms.

### Claim 17 (depends on 15)

The method according to claim 15, including a second stage of adjusting the filling strongly when the filling level is less than a second threshold value less than the first threshold value.

### Claim 18 (depends on 15)

The method according to claim 15, wherein the second threshold value is equal to about 60% of the safety value, and the latency is modified by a value less than or equal to 40 ms.

### Claim 19 (depends on 1)

The method according to claim 1, wherein the filling level of the buffer is determined each time an audio frame is extracted from the buffer or at regular intervals.

### Claim 20 (depends on 1)

The method according to claim 1, wherein the decoder appliance is connected to a plurality of wireless speakers and wherein the predetermined filling level is a global filling level of the buffers of at least a group of speakers from among said speakers.

### Claim 21 (depends on 20)

The method according to claim 20, wherein the speakers that belong to the group are in the proximity of the screen.

### Claim 22 (depends on 20)

The method according to claim 20, wherein the global filling level is the minimum value of the filling levels of the buffers of each of the speakers under consideration.

### Claim 23

A multimedia playback system comprising a decoder connected to a display screen and to at least one wireless speaker, the decoder comprising a demultiplexer having an input connected to a source of audio/video digital data, a first output connected to a video processor circuit arranged to supply video data to the display screen, and a second output connected to an audio processor circuit arranged to supply encoded audio frames to a radio transmitter, the speaker including a radio receiver connected to a decoder circuit arranged to supply a sound playback member with decoded audio frames resulting from decoding the encoded audio frames, the speaker including a buffer for storing audio frames so as to allow latency between receiving encoded audio frames and supplying decoded audio frames, and the system including at least one electronic control unit arranged to perform the method according to any preceding claim.

### Claim 24 (depends on 1)

A decoder appliance arranged to perform the method according to claim 1.

### Claim 25 (depends on 1)

A speaker arranged to perform the method according to claim 1.

### Claim 26 (depends on 1)

A computer program including instructions arranged to perform the method according to claim 1.

### Claim 27 (depends on 26)

A data medium containing a program according to claim 26.
