# US 2023/0057207 A1 · Immersive media compatibility

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0057207 A1
- Document ID: US20230057207A1
- Status: Published application
- Kind: A1
- Application number: 17/892,976
- Filed: 2022-08-22
- Published: 2023-02-23
- Earliest claimed: 2021-08-23
- Assignees: Tencent America LLC
- Applicants: Tencent America LLC
- Inventors: Jun TIAN; Xiaozhong XU; Shan LIU
- Web page: https://patentgrep.com/patent/US20230057207A1

## CPC

- H04S 7/303: {Tracking of listener position or orientation}
- H04S 7/304: {For headphones}
- G02B 27/017: {Head mounted}
- G06F 3/012: {Head tracking input arrangements}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/0346: with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
- G06F 3/165: {Management of the audio stream, e.g. setting of volume, audio stream path}
- G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
- H04N 21/44218: {Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV programme (methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45)}
- H04N 21/816: {involving special video data, e.g 3D video}
- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G06T 2219/2016: Rotation, translation, scaling
- H04R 2499/15: Transducers incorporated in visual displaying devices, e.g. televisions, computer displays, laptops
- H04S 2400/11: Positioning of individual sound objects, e.g. moving airplane, within a sound field (H04S2420/13 takes precedence)
- H04S 2400/13: Aspects of volume control, not necessarily automatic, in stereophonic sound systems
- H04S: STEREOPHONIC SYSTEMS
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K)

## Abstract

Aspects of the disclosure provide methods and apparatuses for audio processing. In some examples, an apparatus for media processing includes processing circuitry. The processing circuitry receives first six degrees of freedom (6 DoF) information associated with a media content for a scene in a media application. The first 6 DoF information includes a first spatial location and a first rotation orientation for rotation about a center at the first spatial location. The processing circuitry determines that a rendering platform for rendering the media content is a three degrees of freedom (3 DoF) platform; and calculates, a revolution orientation of the media content on a sphere centered other than the first spatial location, according to at least the first spatial location. The revolution orientation is 3 DoF information associated with the media content for rendering on the 3 DoF platform.

## Claims

### Claim 1

A method of media processing at a media processing device, comprising:
receiving first six degrees of freedom (6 DoF) information associated with a media content for a scene in a media application, the first 6 DoF information comprising a first spatial location and a first rotation orientation centered at the first spatial location;
determining that a rendering platform for rendering the media content is a three degrees of freedom (3 DoF) platform; and
calculating, a revolution orientation of the media content on a sphere centered other than the first spatial location, according to at least the first spatial location, the revolution orientation being 3 DoF information associated with the media content for rendering on the 3 DoF platform.

### Claim 2 (depends on 1)

The method of claim 1, wherein the first spatial location is described in a 3 dimensional (3D) cartesian coordinate system by three coordinate values respectively associated with three axes of the 3D cartesian coordinate system.

### Claim 3 (depends on 2)

The method of claim 2, further comprising:
receiving second 6 DoF information associated with a user of the media application, the second 6 DoF information comprising a second spatial location and a second rotation orientation centered at the second spatial location; and
calculating, the revolution orientation of the media content on the sphere centered at the second spatial location, according to the first spatial location and the second spatial location.

### Claim 4 (depends on 3)

The method of claim 3, wherein the calculating the revolution orientation of the media content on the sphere centered at the second spatial location further comprises:
calculating a distance between the first spatial location and the second spatial location, the distance being used to adjust a gain for an audio in the media content.

### Claim 5 (depends on 3)

The method of claim 3, wherein the calculating the revolution orientation of the media content on the sphere centered at the second spatial location further comprises:
determining a rotation matrix for rotating on the sphere from a starting point on the sphere to the first spatial location on the sphere; and
calculating the revolution orientation based on the rotation matrix.

### Claim 6 (depends on 3)

The method of claim 3, wherein the calculating the revolution orientation of the media content on the sphere centered at the second spatial location further comprises:
calculating a distance between the first spatial location and the second spatial location;
determining an end point on a unit sphere corresponding to the first spatial location, the unit sphere being centered at the second spatial location, and having a unit radius;
determining a rotation matrix for rotating on the unit sphere from a starting point to the end point; and
calculating the revolution orientation based on the rotation matrix.

### Claim 7 (depends on 1)

The method of claim 1, wherein the first spatial location is described in a 3 dimensional (3D) polar coordinate system by a radial distance, an azimuthal angle, and a polar angle.

### Claim 8 (depends on 7)

The method of claim 7, wherein the calculating, the revolution orientation of the media content on the sphere further comprises:
converting the azimuthal angle to a yaw angle of the revolution orientation; and
converting the polar angle to a pitch angle and a roll angle of the revolution orientation.

### Claim 9 (depends on 1)

The method of claim 1, wherein the media processing device is a media client device, and the method further comprises:
rendering the media content according to the revolution orientation.

### Claim 10 (depends on 1)

The method of claim 1, wherein the media processing device is a media server device, and the method further comprises:
providing the media content associated with the revolution orientation from the media server device to a media client device.

### Claim 11

An apparatus of media processing, comprising processing circuitry configured to:
receive first six degrees of freedom (6 DoF) information associated with a media content for a scene in a media application, the first 6 DoF information comprising a first spatial location and a first rotation orientation centered at the first spatial location;
determine that a rendering platform for rendering the media content is a three degrees of freedom (3 DoF) platform; and
calculate, a revolution orientation of the media content on a sphere centered other than the first spatial location, according to at least the first spatial location, the revolution orientation being 3 DoF information associated with the media content for rendering on the 3 DoF platform.

### Claim 12 (depends on 11)

The apparatus of claim 11, wherein the first spatial location is described in a 3 dimensional (3D) cartesian coordinate system by three coordinate values for three axes.

### Claim 13 (depends on 12)

The apparatus of claim 12, wherein the processing circuitry is configured to:
receive second 6 DoF information associated with a user of the media application, the second 6 DoF information comprising a second spatial location and a second rotation orientation centered at the second spatial location; and
calculate, the revolution orientation of the media content on the sphere centered at the second spatial location, according to the first spatial location and the second spatial location.

### Claim 14 (depends on 13)

The apparatus of claim 13, wherein the processing circuitry is configured to:
calculate a distance between the first spatial location and the second spatial location, the distance being used to adjust a gain for an audio in the media content.

### Claim 15 (depends on 13)

The apparatus of claim 13, wherein the processing circuitry is configured to:
determine a rotation matrix for rotating on the sphere from a starting point on the sphere to the first spatial location on the sphere; and
calculate the revolution orientation based on the rotation matrix.

### Claim 16 (depends on 13)

The apparatus of claim 13, wherein the processing circuitry is configured to:
calculate a distance between the first spatial location and the second spatial location;
determine an end point on a unit sphere corresponding to the first spatial location, the unit sphere being centered at the second spatial location, and having a unit radius;
determine a rotation matrix for rotating on the unit sphere from a starting point to the end point; and
calculate the revolution orientation based on the rotation matrix.

### Claim 17 (depends on 11)

The apparatus of claim 11, wherein the first spatial location is described in a 3 dimensional (3D) polar coordinate system by a radial distance, an azimuthal angle, and a polar angle.

### Claim 18 (depends on 17)

The apparatus of claim 17, wherein the processing circuitry is configured to:
convert the azimuthal angle to a yaw angle of the revolution orientation; and
convert the polar angle to a pitch angle and a roll angle of the revolution orientation.

### Claim 19 (depends on 11)

The apparatus of claim 11, wherein the processing circuitry is configured to:
render the media content according to the revolution orientation.

### Claim 20

A non-transitory computer-readable storage medium storing instructions which when executed by at least one processor cause the at least one processor to perform:
receiving first six degrees of freedom (6 DoF) information associated with a media content for a scene in a media application, the first 6 DoF information comprising a first spatial location and a first rotation orientation centered at the first spatial location;
determining that a rendering platform for rendering the media content is a three degrees of freedom (3 DoF) platform; and
calculating, a revolution orientation of the media content on a sphere centered other than the first spatial location, according to at least the first spatial location, the revolution orientation being 3 DoF information associated with the media content for rendering on the 3 DoF platform.
