# US 2019/0173663 A1 · Audio and video playback system and method for playing audio data applied thereto

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0173663 A1
- Document ID: US20190173663A1
- Status: Published application
- Kind: A1
- Application number: 16/180,073
- Filed: 2018-11-05
- Published: 2019-06-06
- Earliest claimed: 2017-12-01
- Assignees: MStar Semiconductor, Inc.
- Applicants: MStar Semiconductor, Inc.
- Inventors: FU JUN ZHU
- Web page: https://patentgrep.com/patent/US20190173663A1

## CPC

- H04N 21/43076: {of the same content streams on multiple devices, e.g. when family members are watching the same movie on different devices}
- G06F 3/165: {Management of the audio stream, e.g. setting of volume, audio stream path}
- H04L 7/041: {using special codes as synchronising signal}
- H04L 65/764: {at the destination (reformatting of additional data in video clients H04N21/4355)}
- H04N 21/24: Monitoring of processes or resources, e.g. monitoring of server load, available bandwidth, upstream requests {(monitoring of server performance or load G06F11/34; arrangements for observation, testing or troubleshooting for broadcast or for distribution combined with broadcast H04H20/12)}
- H04N 21/43637: {involving a wireless protocol, e.g. Bluetooth®, RF or wireless LAN [IEEE 802.11] (arrangements for wireless networking or broadcasting of information in indoor or near-field type systems H04B10/114)}
- H04N 21/4392: {involving audio buffer management}
- H04L 2007/045: {Fill bit or bits, idle words}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

An audio and video playback system includes an audio and video playback device having a local audio device, and a secondary audio device. A method for playing audio data includes: allocating a local audio buffer space and a secondary audio buffer space to the local audio device and the secondary audio device, respectively; processing obtained multimedia data to generate local audio data and secondary audio data; writing the local audio data and the secondary audio data to the local audio buffer space and the secondary audio buffer space, respectively; reading the local audio data and the secondary audio data buffered in the local audio buffer space and the secondary audio buffer space to the local audio device and the secondary audio device, to have the local audio device and the secondary audio device play the local audio data and the secondary audio data, respectively.

## Claims

### Claim 1

A method for playing audio data, the method applied to an audio and video playback system, the audio and video playback system comprising a secondary audio device and an audio and video playback device having a local audio device, the method comprising:
allocating a local audio buffer space and a secondary audio buffer space to the local audio device and the secondary audio device, respectively;
processing obtained multimedia data to generate local audio data and secondary audio data;
writing the local audio data and the secondary audio data to the local audio buffer space and the secondary audio buffer space, respectively; and
reading the local audio data and the secondary audio data buffered in the local audio buffer space and the secondary audio buffer space to the local audio device and the secondary audio device, respectively, to have the local audio device and the secondary audio device play the local audio data and the secondary audio data, respectively.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the local audio buffer space and the secondary audio buffer space are each a ring buffer space, such that while the local audio data and the secondary audio data buffered in the local audio buffer space and the secondary audio buffer space are read to the local audio device and the secondary audio device, the local audio data and the secondary audio data are continuously further written to the local audio buffer space and the secondary audio buffer space, respectively.

### Claim 3 (depends on 2)

The method according to claim 2, further comprising:
determining whether the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device are synchronous;
when it is determined that the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device are asynchronous, executing a synchronization mechanism to synchronize the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device are synchronous.

### Claim 4 (depends on 3)

The method according to claim 3, wherein the step of determining whether the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device are synchronous further comprises:
acquiring delay periods of the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device, respectively; and
determining, according to the respective delay periods of the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device, whether the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device are synchronous.

### Claim 5 (depends on 4)

The method according to claim 4, wherein the step of executing the synchronization mechanism to synchronize the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device further comprises:
determining a difference between the delay periods of the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device;
generating a null packet having a corresponding size according to the difference between the delay periods of the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device; and
writing the null packet to the local audio buffer space or the secondary audio buffer space corresponding to the local audio device or the secondary audio device having the smaller delay period, to cause the local audio device or the secondary audio device to output the null packet when outputting the local audio data or the secondary audio data, thereby synchronizing the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device.

### Claim 6 (depends on 5)

The method according to claim 5, wherein the step of generating the null packet having the corresponding size according to the difference between the delay periods of the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device comprises:
determining whether the difference between the delay periods of the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device is greater than a predetermined value; and
when the difference between the delay periods of the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device is greater than the predetermined value, generating the null packet having the corresponding size according to the difference between the delay periods of the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device.

### Claim 7 (depends on 4)

The method according to claim 4, wherein the step of executing the synchronization mechanism to synchronize the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device further comprises:
determining a difference between the delay periods of the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device;
obtaining a corresponding data size according to the difference between the delay periods of the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device; and
discarding data having the corresponding data size from the local audio buffer space or the secondary audio buffer space corresponding to the local audio device or the secondary audio device having the greater delay period, thereby synchronizing the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device.

### Claim 8 (depends on 7)

The method according to claim 7, wherein the step of obtaining the corresponding data size according to the difference between the delay periods of the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device further comprises:
determining whether the difference between the delay periods of the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device is greater than a predetermined value; and
when the difference between the delay periods of the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device is greater than the predetermined value, obtaining the corresponding data size according to the difference between the delay periods of the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device.

### Claim 9 (depends on 1)

The method according to claim 1, wherein the audio and video playback device is a television or a television system consisting of a set-top box (STB) and a television, the local audio device is a local speaker built in the television, and the secondary audio device is an external acoustic enclosure connected to the television.

### Claim 10 (depends on 9)

The method according to claim 9, wherein the secondary audio device and the television are connected by means of USB wireless connection.

### Claim 11

An audio and video playback system, comprising:
an audio and video playback device, for playing multimedia data, the audio and video playback device comprising a local audio device, a processor, a memory and a buffer unit; and
a secondary audio device, connected to the audio and video playback device;
wherein, an audio device management program is stored in the memory of the audio and video playback device; when the audio device management program is executed, the audio and video playback device controls the local audio device and the secondary audio device according to the audio device management program to perform a collaborated playback task to play audio data in the multimedia data.

### Claim 12 (depends on 11)

The audio and video playback system according to claim 11, wherein when the audio and video playback device controls the local audio device and the secondary audio device according to the audio device management program to perform the collaborated playback task to play audio data in the multimedia data, the processor in the audio and video playback device performs steps of:
allocating a local audio buffer space and a secondary audio buffer space to the local audio device and the secondary audio device, respectively;
processing the obtained multimedia data to generate local audio data and secondary audio data;
writing the local audio data and the secondary audio data to the local audio buffer space and the secondary audio buffer space, respectively; and
reading the local audio data and the secondary audio data buffered in the local audio buffer space and the secondary audio buffer space to the local audio device and the secondary audio device, respectively, to have the local audio device and the secondary audio device play the local audio data and the secondary audio data, respectively.

### Claim 13 (depends on 12)

The audio and video playback system according to claim 12, wherein the local audio buffer space and the secondary audio buffer space are each a ring buffer space, such that while the local audio data and the secondary audio data buffered in the local audio buffer space and the secondary audio buffer space are read to the local audio device and the secondary audio device, the local audio data and the secondary audio data are continuously further written to the local audio buffer space and the secondary audio buffer space, respectively.

### Claim 14 (depends on 13)

The audio and video playback system according to claim 13, wherein when the audio and video playback controls the local audio device and the secondary audio device to collaboratively play the audio data in the multimedia data, the processor further performs steps of:
determining whether the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device are synchronous;
when it is determined that the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device are asynchronous, executing a synchronization mechanism to synchronize the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device are synchronous.

### Claim 15 (depends on 14)

The audio and video playback system according to claim 14, wherein the step of determining whether the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device are synchronous further comprises:
acquiring delay periods of the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device, respectively; and
determining, according to the respective delay periods of the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device, whether the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device are synchronous.

### Claim 16 (depends on 15)

The audio and video playback system according to claim 15, wherein the step of the step of executing the synchronization mechanism to synchronize the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device further comprises:
determining a difference between the delay periods of the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device;
generating a null packet having a corresponding size according to the difference between the delay periods of the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device; and
writing the null packet to the local audio buffer space or the secondary audio buffer space corresponding to the local audio device or the secondary audio device having the smaller delay period, to cause the local audio device or the secondary audio device to output the null packet when outputting the local audio data or the secondary audio data, thereby synchronizing the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device.

### Claim 17 (depends on 15)

The audio and video playback system according to claim 15, wherein the step of executing the synchronization mechanism to synchronize the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device further comprises:
determining a difference between the delay periods of the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device;
obtaining the corresponding data size according to the difference between the delay periods of the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device; and
discarding data having the corresponding data size from the local audio buffer space or the secondary audio buffer space corresponding to the local audio device or the secondary audio device having the greater delay period, thereby synchronizing the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device.

### Claim 18 (depends on 11)

The audio and video playback system according to claim 11, wherein the audio device management program further comprises an audio device playback mode selection function; when the audio device playback mode selection function is executed, the audio and video playback device drives the local audio device and/or the secondary audio device according to the selected audio device playback mode to play the audio data in the multimedia data.

### Claim 19 (depends on 11)

The audio and video playback system according to claim 11, wherein the audio and video playback device is a television or a television system consisting of a set-top box (STB) and a television, the local audio device is a local speaker built in the television, and the secondary audio device is an external acoustic enclosure connected to the television.

### Claim 20 (depends on 11)

The audio and video playback system according to claim 11, wherein the secondary audio device and the television are connected by means of USB wireless connection.
