# US 2021/0084425 A1 · Representation of contextual information by projecting different participants' audio from different positions in a 3D soundscape

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0084425 A1
- Document ID: US20210084425A1
- Status: Published application
- Kind: A1
- Application number: 17/094,589
- Filed: 2020-11-10
- Published: 2021-03-18
- Earliest claimed: 2016-12-30
- Assignees: Akamai Technologies, Inc.
- Applicants: Akamai Technologies, Inc.
- Inventors: Abhijit C. Mehta; Charles E. Gero; Thomas Houman
- Web page: https://patentgrep.com/patent/US20210084425A1

## CPC

- H04S 1/002: {Non-adaptive circuits, e.g. manually adjustable or static, for enhancing the sound image or the spatial distribution (control circuits for electronic adaptation of the sound field H04S7/30)}
- G06F 3/165: {Management of the audio stream, e.g. setting of volume, audio stream path}
- H04L 65/403: Arrangements for multi-party communication, e.g. for conferences (data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15)
- H04L 65/764: {at the destination (reformatting of additional data in video clients H04N21/4355)}
- H04N 7/15: Conference systems
- H04S 2400/11: Positioning of individual sound objects, e.g. moving airplane, within a sound field (H04S2420/13 takes precedence)
- H04S: STEREOPHONIC SYSTEMS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

An overlay network platform facilitates a multi-party conference. End users participate in the conference using client-based web browser software. According to this disclosure, an enhanced “audio” experience for a user participating in the conference is provided by rendering different participants' audio in the conference at different positions in a three-dimensional (3D) space. In operation, and given a set of audio streams that comprise the conference and that are being received, a three-dimensional (3D) position for each audio stream in a 3D soundscape is computed. In one embodiment, the 3D position for each audio stream is computed by identifying a target area region of space, and then determining the 3D position for each stream in the target area region of space. The relative 3D positions may be determined based on contextual information related to the content of an audio stream, as well as a computed “relevance” of one stream versus another. The streams are then rendered to the listener based on the computed 3D positions.

## Claims

### Claim 1

A method carried out at a given client machine, the given client machine being one of a set of client machines participating in a multi-party conference using an overlay network platform, the multi-party conference comprising a set of audio streams received from multiple users each associated with a respective client machine, comprising:
computing a position in a soundscape for each of the received audio streams by identifying a target area region of space in front of the user of the given client machine, and identifying a relative position of each of a set of remote users in the multi-party conference as follows: (a) applying a function to a unique identifier associated with the remote user of the set of remote users to generate a number, (b) based on the number generated for each of the set of remote users, assigning each of the remote users a relative left-right position within the target area region of space in front of the user;
rendering the audio streams of the remote users at the left-right position computed;
as the multi-party conference continues, selectively adjusting a front-back position of at least one of the remote users relative to the assigned left-right positions based on a context associated with a given user or a given audio stream; and
re-rendering the audio streams of the remote users including at the adjusted front-back position.

### Claim 2 (depends on 1)

The method as described in claim 1 wherein, with respect to a location of a listener, the target area region is defined by a 45° cone.

### Claim 3 (depends on 1)

The method as described in claim 1 wherein the position of the at least one of the remote users also is based on a position on a display screen where a video of the speaker that generates that stream is located.

### Claim 4 (depends on 1)

The method as described in claim 1 wherein, with respect to the relative left-right positions, audio streams of the remote users are evenly spaced within the target area region of space.

### Claim 5 (depends on 4)

The method as described in claim 4 wherein the audio streams are evenly spaced by applying a hash function to the unique identifier, the unique identifier being the remote user's name.

### Claim 6 (depends on 1)

The method as described in claim 1 further including re-adjusting the front-back position of the at least one of the remote users relative to the assigned left-right positions.

### Claim 7 (depends on 1)

The method as described in claim 1 wherein the audio streams are rendered in a browser.

### Claim 8 (depends on 7)

The method as described in claim 7 wherein the browser renders the audio streams using an application programming interface (API).

### Claim 9 (depends on 8)

The method as described in claim 8 wherein the API is a Hypertext Markup Language Version 5 (HTML5) Web Audio Application Programming Interface.
