# US 11,910,344 B2 · Conference audio management

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,910,344 B2
- Document ID: US11910344B2
- Status: Granted
- Kind: B2
- Application number: 16/249,754
- Filed: 2019-01-16
- Published: 2019-05-16
- Granted: 2024-02-20
- Earliest claimed: 2015-04-05
- Assignees: QUALCOMM Incorporated
- Applicants: QUALCOMM Incorporated
- Inventors: Venkatraman S. Atti; Daniel J. Sinder; Nikolai Leung; Vivek Rajendran
- Web page: https://patentgrep.com/patent/US11910344B2

## CPC

- H04W 56/0045: {compensating for timing error by altering transmission time}
- H04L 43/087: Jitter
- H04L 65/65: Network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP]
- H04M 3/562: {where the conference facilities are distributed}
- H04M 3/568: {audio processing specific to telephonic conferencing, e.g. spatial distribution, mixing of participants (echo suppression in two-way loud-speaking telephone systems H04M9/02; sound field processing per se H04S7/30)}
- H04R 29/007: {for public address systems (public address systems per se H04R27/00)}
- H04S 7/00: Indicating arrangements; Control arrangements, e.g. balance control
- H04S 2420/01: Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]
- H04W 80/04: Network layer protocols, e.g. mobile IP [Internet Protocol]
- H04W 88/02: Terminal devices
- H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72)
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- H04M: TELEPHONIC COMMUNICATION (circuits for controlling other apparatus via a telephone cable and not involving telephone switching apparatus G08)
- H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K)
- H04S: STEREOPHONIC SYSTEMS

## Abstract

A method for managing audio during a conference includes steering, at a spatial steering processor of a mobile device, first decoded audio to be projected from a speaker at a first angle. The method also includes steering, at the spatial steering processor, second decoded audio to be projected from the speaker at a second angle. The first decoded audio corresponds to a decoded version of a first audio stream from a first device associated with a first participant of the conference, and the second decoded audio corresponds to a decoded version of a second audio stream from a second device associated with a second participant of the conference. The first decoded audio is synchronized with the second decoded audio.

## Claims

### Claim 1

A mobile device for managing audio during a conference, the mobile device comprising:
a first decoder configured to decode a first audio stream, based upon a determination to prioritize the first audio stream, from a first device associated with a first participant of the conference to generate a first decoded audio, the determination to prioritize the first audio stream being based upon a frame energy, or an active level;
a second decoder configured to decode a second audio stream from a second device associated with a second participant of the conference to generate a second decoded audio;
a spatial steering processor configured to:
obtain user input of a user of the mobile device, the user input including a first user indication that the first participant associated with the first device is a preferred participant;
responsive to the user input, (a) provide a control signal to a network device to adjust a first bit rate, in a manner indicated by user preference, of the first audio stream and (b) increase a first gain of the first decoded audio;
steer the first decoded audio to be projected from a speaker at a first angle, the first angle based upon a second user indication of an angle at which audio is to be projected, the first decoded audio corresponding to a decoded version of the first audio stream from the first device;
steer the second decoded audio to be projected from the speaker at a second angle, the second decoded audio corresponding to a decoded version of the second audio stream from a second device associated with a second participant of the conference, and the first decoded audio synchronized with the second decoded audio;
receive a signal indicating detection of head movement associated with the user of the mobile device, wherein the user of the mobile device is distinct from the first participant and second participant; and
shift the first angle and the second angle by a shift amount in response to receiving the signal; and
the speaker configured to:
project the first decoded audio at the shifted first angle; and
project the second decoded audio at the shifted second angle.

### Claim 2 (depends on 1)

The mobile device of claim 1, wherein the spatial steering processor is configured to:
select, based on determining that the first participant corresponds to the preferred participant, a first head-related transfer function (HRTF) for the first device, the first HRTF corresponding to the first angle and a first gain;
select, based on determining that the second participant does not correspond to the preferred participant, a second HRTF for the second device, the second HRTF corresponding to the second angle and a second gain, wherein the first gain is higher than the second gain;
apply the first HRTF to the first decoded audio to steer the first decoded audio; and
apply the second HRTF to the second decoded audio to steer the second decoded audio.

### Claim 3 (depends on 1)

The mobile device of claim 1, further comprising:
a first de-jitter buffer configured to receive the first audio stream from the first device;
a second de-jitter buffer configured to receive the second audio stream from the second device; and
a delay controller configured to generate a control signal, the control signal provided to the first de-jitter buffer and to the second de-jitter buffer to synchronize first buffered audio that is outputted from the first de-jitter buffer with second buffered audio that is output from the second de-jitter buffer,
wherein the first decoded audio is synchronized with the second decoded audio based on the synchronization of the first buffered audio and the second buffered audio.

### Claim 4 (depends on 3)

The mobile device of claim 3, wherein the delay controller is further configured to:
compare a first time stamp of the first audio stream with a second time stamp of the second audio stream, the first time stamp and the second time stamp based on a common clock source; and
determine a time difference between the first time stamp and the second time stamp.

### Claim 5 (depends on 3)

The mobile device of claim 3, wherein the first buffered audio is synchronized with the second buffered audio prior to performing decoding operations to output a first signal associated with the first buffered audio and a second signal associated with the second buffered audio.

### Claim 6 (depends on 4)

The mobile device of claim 4, wherein the control signal indicates to the first de-jitter buffer to delay outputting the first buffered audio by the time difference if the first time stamp indicates an earlier time than the second time stamp.

### Claim 7 (depends on 4)

The mobile device of claim 4, wherein the control signal indicates to the second de-jitter buffer to delay outputting the second buffered audio by the time difference if the second time stamp indicates an earlier time than the first time stamp.

### Claim 8 (depends on 1)

The mobile device of claim 1, further comprising an antenna configured to receive the first audio stream.

### Claim 9 (depends on 1)

The mobile device of claim 1, further comprising a modem that includes the spatial steering processor.

### Claim 10 (depends on 1)

The mobile device of claim 1, wherein the mobile device, the first device, and the second device each comprise a user equipment (UE) that is compatible with a Third Generation Partnership Project (3GPP) standard.

### Claim 11

A method for managing audio during a conference, the method comprising:
decoding, at a first decoder of a mobile device, a first audio stream, based upon a determination to prioritize the first audio stream, received from a first device associated with a first participant of the conference to generate a first decoded audio, the determination to prioritize the first audio stream being based upon a frame energy or an active level;
decoding, at a second decoder of the mobile device, a second audio stream received from a second device associated with a second participant of the conference to generate a second decoded audio;
obtaining user input of a user of the mobile device, the user input including a first user indication that the first participant associated with the first device is a preferred participant;
responsive to the user input, (a) providing a control signal to a network device to adjust a first bit rate, in a manner indicated by user preference, of the first audio stream and (b) increasing a first gain of the first decoded audio;
steering, at a spatial steering processor, the first decoded audio to be projected from a speaker at a first angle, the first angle based upon a second user indication of an angle at which audio is to be projected, the first decoded audio corresponding to a decoded version of the first audio stream from the first device;
steering, at the spatial steering processor, the second decoded audio to be projected from the speaker at a second angle, the second decoded audio corresponding to a decoded version of the second audio stream from a second device associated with a second participant of the conference, and the first decoded audio synchronized with the second decoded audio;
receiving a signal indicating detection of head movement associated with the user of the mobile device, wherein the user of the mobile device is distinct from the first participant and second participant; and
shifting the first angle and the second angle by a shift amount in response to receiving the signal.

### Claim 12 (depends on 11)

The method of claim 11, further comprising:
applying a first head-related transfer function (HRTF) to the first decoded audio to steer the first decoded audio; and
applying a second HRTF to the second decoded audio to steer the second decoded audio.

### Claim 13 (depends on 11)

The method of claim 11, further comprising:
receiving, at a first buffer of the mobile device, the first audio stream from the first device;
receiving, at a second buffer of the mobile device, the second audio stream from the second device; and
generating a control signal at a delay controller for the mobile device, the control signal provided to the first buffer and to the second buffer to synchronize first buffered audio that is outputted from the first buffer with second buffered audio that is output from the second buffer,
wherein the first decoded audio is synchronized with the second decoded audio based on the synchronization of the first buffered audio and the second buffered audio.

### Claim 14 (depends on 13)

The method of claim 13, further comprising:
comparing, at the mobile device, a first time stamp of the first audio stream with a second time stamp of the second audio stream, the first time stamp and the second time stamp based on a common clock source; and
determining, at the mobile device, a time difference between the first time stamp and the second time stamp.

### Claim 15 (depends on 13)

The method of claim 13, wherein the first buffer comprises a first de-jitter buffer, and wherein the second buffer comprises a second de-jitter buffer.

### Claim 16 (depends on 14)

The method of claim 14, wherein the control signal indicates to the first buffer to delay outputting the first buffered audio by the time difference if the first time stamp indicates an earlier time than the second time stamp.

### Claim 17 (depends on 14)

The method of claim 14, wherein the control signal indicates to the second buffer to delay outputting the second buffered audio by the time difference if the second time stamp indicates an earlier time than the first time stamp.

### Claim 18 (depends on 11)

The method of claim 11, wherein the mobile device, the first device, and the second device each comprise a user equipment (UE) that is compatible with a Third Generation Partnership Project (3GPP) standard.

### Claim 19 (depends on 11)

The method of claim 11, wherein the first audio stream is received via an antenna of the mobile device.

### Claim 20 (depends on 11)

The method of claim 11, wherein the spatial steering processor is included in a modem of the mobile device.

### Claim 21

A non-transitory computer-readable medium comprising instructions for managing audio during a conference, the instructions, when executed by a spatial steering processor in a mobile device, cause the spatial steering processor to perform operations comprising:
decoding, at a first decoder of a mobile device, a first audio stream, based upon a determination to prioritize the first audio stream, received from a first device associated with a first participant of the conference to generate a first decoded audio, the determination to prioritize the first audio stream being based upon a frame energy or an active level;
decoding, at a second decoder of the mobile device, a second audio stream received from a second device associated with a second participant of the conference to generate a second decoded audio;
obtaining user input of a user of the mobile device, the user input including a first user indication that the first participant associated with the first device is a preferred participant;
responsive to the user input, (a) providing a control signal to a network device to adjust a first bit rate, in a manner indicated by user preference, of the first audio stream and (b) increasing a first gain of the first decoded audio;
steering the first decoded audio to be projected from a speaker at a first angle, the first angle based upon a second user indication of an angle at which audio is to be projected, the first decoded audio corresponding to a decoded version of the first audio stream from the first device;
steering the second decoded audio to be projected from the speaker at a second angle, the second decoded audio corresponding to a decoded version of the second audio stream from a second device associated with a second participant of the conference, and the first decoded audio synchronized with the second decoded audio; and
receiving a signal indicating detection of head movement associated with the user of the mobile device, wherein the user of the mobile device is distinct from the first participant and second participant; and
shifting the first angle and the second angle by a shift amount in response to receiving the signal.

### Claim 22 (depends on 21)

The non-transitory computer-readable medium of claim 21, wherein the operations further comprise:
applying a first head-related transfer function (HRTF) to the first decoded audio to steer the first decoded audio; and
applying a second HRTF to the second decoded audio to steer the second decoded audio.

### Claim 23 (depends on 21)

The non-transitory computer-readable medium of claim 21, wherein the mobile device, the first device, and the second device each comprise a user equipment (UE) that is compatible with a Third Generation Partnership Project (3GPP) standard.

### Claim 24

An apparatus for managing audio during a conference, the apparatus comprising:
means for decoding, at a first decoder of a mobile device, a first audio stream received from a first device, based upon a determination to prioritize the first audio stream, associated with a first participant of the conference to generate a first decoded audio, the determination to prioritize the first audio stream being based upon a frame energy or an active level;
means for decoding, at a second decoder of the mobile device, a second audio stream received from a second device associated with a second participant of the conference to generate a second decoded audio;
means for obtaining user input of a user of the mobile device, the user input including a first user indication that the first participant associated with the first device is a preferred participant;
means for providing, responsive to the first user indication, a control signal to a network device to adjust a first bit rate, in a manner indicated by user preference, of the first audio stream based upon user preferences;
means for increasing, responsive to the first user indication, a first gain of the first decoded audio;
means for steering the first decoded audio to be projected from means for projecting at a first angle, the first angle based upon a second user indication of an angle at which audio is to be projected, and steering the second decoded audio to be projected from the means for projecting at a second angle, the first decoded audio corresponding to a decoded version of the first audio stream from the first device, the second decoded audio corresponding to a decoded version of the second audio stream from a second device associated with a second participant of the conference, and the first decoded audio synchronized with the second decoded audio;
the means for projecting; and
means for receiving a signal indicating detection of head movement associated with the user of the mobile device, wherein the user of the mobile device is distinct from the first participant and second participant; and
means for shifting the first angle and the second angle by a shift amount in response to receiving the signal.

### Claim 25 (depends on 24)

The apparatus of claim 24, wherein the means for steering the first decoded audio and the second decoded audio is included in a modem of a mobile device.

### Claim 26 (depends on 25)

The apparatus of claim 25, wherein the first audio stream is received via an antenna of the mobile device.