# US 2024/0406659 A1 · Spatial audio

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0406659 A1
- Document ID: US20240406659A1
- Status: Published application
- Kind: A1
- Application number: 18/670,832
- Filed: 2024-05-22
- Published: 2024-12-05
- Earliest claimed: 2023-06-02
- Assignees: Nokia Technologies Oy
- Applicants: Nokia Technologies Oy
- Inventors: Jussi Artturi LEPPÄNEN; Sujeet Shyamsundar MATE; Arto Juhani LEHTINIEMI
- Web page: https://patentgrep.com/patent/US20240406659A1

## CPC

- H04S 7/303: {Tracking of listener position or orientation}
- H04S 7/304: {For headphones}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/04815: Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object
- G06F 3/16: Sound input; Sound output (speech processing G10L)
- H04S 7/40: {Visual indication of stereophonic sound image}
- H04S 2400/11: Positioning of individual sound objects, e.g. moving airplane, within a sound field (H04S2420/13 takes precedence)
- H04S: STEREOPHONIC SYSTEMS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

An apparatus comprising: means for receiving a data structure for controlling rendering of a sound scene to a user of the apparatus, wherein the data structure specifies one or more positions for initial rendering a new sound object associated with a different user; means for selecting one of the one or more positions specified by the data structure, wherein the means for selecting one of the one or more positions specified by the data structure is configured to select a specified position in dependence upon at least a position of the user; and means for rendering the new sound object at the selected position.

## Claims

### Claim at position 1

1-15. (canceled)

### Claim 16

An apparatus comprising:
at least one processor; and
at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to:
receive a data structure for controlling rendering of a sound scene to a user of the apparatus, wherein the data structure specifies one or more positions for initial rendering of a new sound object associated with a different user;
select one of the one or more positions specified by the data structure in dependence upon at least a position of the user; and
render the new sound object at the selected position.

### Claim 17 (depends on 16)

An apparatus as claimed in claim 16, wherein the rendering enables the user and the different user to share a virtual three-dimensional space, wherein the sound scene is a three-dimensional sound scene that has a fixed relationship to the virtual three-dimensional space.

### Claim 18 (depends on 17)

An apparatus as claimed in claim 17, wherein the rendering enables the user to change position and/or orientation of the user within the virtual three-dimensional space.

### Claim 19 (depends on 17)

An apparatus as claimed in claim 17, wherein the rendering enables, after the initial rendering of the sound object associated with the different user at the selected position, the different user to change a position and/or orientation at which the sound object associated with the different user is rendered within the virtual three-dimensional space.

### Claim 20 (depends on 16)

An apparatus as claimed in claim 16, wherein the apparatus is further caused to display a visual object at the selected position at which the new sound object is rendered.

### Claim 21 (depends on 16)

An apparatus as claimed in claim 16, wherein the sound object is a live social audio object associated with the different user and comprises at least live speech of the different user.

### Claim 22 (depends on 16)

An apparatus as claimed in claim 16, wherein selecting one of the one or more positions specified by the data structure further comprises selecting a specified position in dependence upon one or more of:
an orientation of the user;
a state of the sound scene;
estimated audibility to the user of the new sound object at the one or more specified positions;
estimated visibility to the user of the one or more specified positions;
a direct line of hearing between a position of the user and the one or more specified positions;
a direct line of visibility between a position of the user and the one or more specified positions;
whether the position of the user and any of the one or more specified positions share a defined acoustic space in a three-dimensional sound scene;
whether the position of the user and any of the one or more specified positions share a defined physical space in a virtual three-dimensional space;
whether one of the one or more specified positions is specified as a default position;
a distance or weighted distance between a position of the user and the one or more specified positions;
sound attenuation attributes of a virtual three-dimensional space between a position of the user and the one or more specified positions.

### Claim 23 (depends on 16)

An apparatus as claimed in claim 16, wherein selecting one of the one or more positions specified by the data structure comprises selecting a specified position in dependence upon at least one parameter specified in the data structure.

### Claim 24 (depends on 23)

An apparatus as claimed in claim 23, wherein selecting one of the one or more positions specified by the data structure comprises selecting a sub-set of positions from the one or more positions specified by the data structure in dependence upon the at least one parameter specified in the data structure and comprises selecting the selected position from the sub-set of positions in dependence upon a distance or weighted distance between a position of the user and the one or more specified positions in the sub-set.

### Claim 25 (depends on 16)

An apparatus as claimed in claim 16, wherein rendering the new sound object at the selected position comprises creating a sound scene state based on which sound is to be rendered to the user at a current position and/or orientation of the user in a virtual three-dimensional space.

### Claim 26 (depends on 16)

An apparatus as claimed in claim 16, wherein the data structure is a data structure for controlling spatial rendering of a sound object to a user and the data structure comprises:
a first element configured to specify a sound object;
a second element configured to specify multiple alternative positions for initial rendering the sound object and at least one parameter for controlling selection of a position for initial rendering the sound object from the specified multiple alternative positions for initial rendering the sound object.

### Claim 27 (depends on 26)

An apparatus as claimed in claim 26, wherein the first element is a sub-element of the second element or the second element is a sub-element of the first element.

### Claim 28 (depends on 16)

An apparatus as claimed in claim 16, configured as a head-mounted apparatus.

### Claim 29

A method comprising:
receiving a data structure for controlling rendering of a sound scene to a user of the apparatus, wherein the data structure specifies one or more positions for initial rendering a new sound object associated with a different user;
selecting one of the one or more positions specified by the data structure in dependence upon at least a position of the user; and
rendering the new sound object at the selected position.

### Claim 30 (depends on 29)

The method of claim 29, wherein the rendering enables the user and the different user to share a virtual three-dimensional space, wherein the sound scene is a three-dimensional sound scene that has a fixed relationship to the virtual three-dimensional space.

### Claim 31 (depends on 30)

The method of claim 30, wherein the rendering enables the user to change position and/or orientation of the user within the virtual three-dimensional space.

### Claim 32 (depends on 30)

The method of claim 30, wherein the rendering enables, after the initial rendering of the sound object associated with the different user at the selected position, the different user to change a position and/or orientation at which the sound object associated with the different user is rendered within the virtual three-dimensional space.

### Claim 33 (depends on 29)

The method of claim 29, further comprising:
displaying a visual object at the selected position at which the new sound object is rendered.

### Claim 34 (depends on 29)

The method of claim 29, wherein the sound object is a live social audio object associated with the different user and comprises at least live speech of the different user.

### Claim 35

A non-transitory computer readable medium comprising program instructions that, when executed by one or more processors of an apparatus, causes the apparatus to perform at least the following:
using a data structure to control rendering of a sound scene to a user of the apparatus, wherein the data structure specifies one or more positions for initial rendering a new sound object associated with a different user;
selecting one of the one or more positions specified by the data structure in dependence upon at least a position of the user; and
rendering the new sound object at the selected position.
