# US 12,003,693 B2 · Object collision data for virtual camera in virtual interactive scene defined by streamed media data

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,003,693 B2
- Document ID: US12003693B2
- Status: Granted
- Kind: B2
- Application number: 17/654,023
- Filed: 2022-03-08
- Published: 2022-09-15
- Granted: 2024-06-04
- Earliest claimed: 2021-03-10
- Assignees: QUALCOMM Incorporated
- Applicants: QUALCOMM Incorporated
- Inventors: Imed Bouazizi; Thomas Stockhammer
- Web page: https://patentgrep.com/patent/US12003693B2

## CPC

- H04N 13/178: Metadata, e.g. disparity information
- H04N 21/816: {involving special video data, e.g 3D video}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06T 13/40: of characters, e.g. humans, animals or virtual beings
- G06T 17/10: Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes
- G06T 17/20: Finite element generation, e.g. wire-frame surface description, {tesselation}
- H04N 5/2224: {related to virtual studio applications}
- H04N 13/117: the virtual viewpoint locations being selected by the viewers or determined by viewer tracking
- H04N 13/275: from three-dimensional [3D] object models, e.g. computer-generated stereoscopic image signals
- H04N 21/21805: {enabling multiple viewpoints, e.g. using a plurality of cameras}
- H04N 21/431: Generation of visual interfaces {for content selection or interaction}; Content or additional data rendering
- H04N 21/44012: {involving rendering scenes according to scene graphs, e.g. MPEG-4 scene graphs}
- H04N 21/47205: {for manipulating displayed content, e.g. interacting with MPEG-4 objects, editing locally}
- G06T 2210/12: Bounding box
- G06T 2210/21: Collision detection, intersection
- H04N 21/26258: {for generating a list of items to be played back in a given order, e.g. playlist, or scheduling item distribution according to such list (retrieval of multimedia data based on playlists G06F16/40)}
- H04N 21/8456: {by decomposing the content in the time domain, e.g. in time segments}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

An example device for retrieving media data includes a memory configured to store media data; and one or more processors implemented in circuitry and configured to execute a presentation engine, the presentation engine being configured to: receive streamed media data representing a virtual three-dimensional scene including at least one virtual solid object; receive object collision data representing boundaries of the at least one virtual solid object; receive camera movement data from a user requesting that the virtual camera move through the at least one virtual solid object; and using the object collision data, prevent the virtual camera from passing through the at least one virtual solid object in response to the camera movement data.

## Claims

### Claim 1

A method of retrieving media data, the method comprising:
receiving, by a presentation engine, streamed media data representing a virtual three-dimensional (3D) scene;
receiving, by the presentation engine, an extension data structure associated with the virtual 3-D scene, the extension data structure including:
first object information comprising a vertex coordinate indicative of at least one virtual cuboid object in the virtual 3D scene, and
animation information indicative of an animation triggered based on interaction of a virtual camera with the at least one virtual cuboid object in the virtual 3D scene;
receiving, by the presentation engine, camera movement data from a user requesting that the virtual camera move through the at least one virtual cuboid object; and
using the first object information, updating, by the presentation engine, a location of the virtual camera to ensure the virtual camera remains outside of the at least one virtual cuboid object in response to the camera movement data.

### Claim 2 (depends on 1)

The method of claim 1, wherein updating the location of the virtual camera comprises preventing the virtual camera from passing through the at least virtual cuboid object.

### Claim 3 (depends on 1)

The method of claim 1, wherein receiving the extension data structure comprises receiving an MPEG_extension.

### Claim 4 (depends on 3)

The method of claim 3, wherein the MPEG_extension further includes data defining at least one 3D mesh for the at least one virtual cuboid object.

### Claim 5 (depends on 4)

The method of claim 4, wherein the MPEG_extension includes data defining at least one of boundaries of the at least one 3D mesh for the at least one virtual cuboid object or a material for the 3D mesh.

### Claim 6 (depends on 1)

The method of claim 1, wherein receiving the first object information comprises receiving data including one or more of:
boundaries data representing one or more collision boundaries of the at least one virtual cuboid object;
static data representing whether the at least one virtual cuboid object is affected by collisions; or
material data representing how colliding objects interact with the at least one virtual cupid object.

### Claim 7 (depends on 1)

The method of claim 1, wherein the at least one virtual cuboid object comprises one of a virtual wall, a virtual chair, or a virtual table.

### Claim 8 (depends on 1)

The method of claim 1, wherein the streamed media data comprises glTF 2.0 media data.

### Claim 9 (depends on 1)

The method of claim 1, wherein receiving the streamed media data comprises requesting the streamed media data from a retrieval unit via an application programming interface (API).

### Claim 10 (depends on 1)

The method of claim 1, wherein the extension data structure is included in an MPEG scene description.

### Claim 11

A device for retrieving media data, the device comprising:
a memory configured to store media data; and
one or more processors implemented in circuitry and configured to execute a presentation engine configured to:
receive streamed media data representing a virtual three-dimensional OD) scene;
receive an extension data structure associated with the virtual 3-D scene, the extension data structure including:
first object information comprising a vertex coordinate indicative of at least one virtual cuboid object in the virtual 3D scene, and
animation information indicative of an animation to be presented based on interaction of a virtual camera with the at least one virtual cuboid object in the virtual 3D scene;
receive camera movement data from a user requesting that the virtual camera move through the at least one virtual cuboid object; and
using the first object information, update a location of the virtual camera to ensure the virtual camera remains outside of the at least one virtual cuboid object in response to the camera movement data.

### Claim 12 (depends on 11)

The device of claim 11, wherein to update the location of the virtual camera, the presentation engine is configured to prevent the virtual camera from passing through the at least virtual cuboid object.

### Claim 13 (depends on 11)

The device of claim 11, wherein to receive the extension data structure, the presentation engine is configured to receive an MPEG_extension.

### Claim 14 (depends on 13)

The device of claim 13, wherein the MPEG_extension further includes data defining at least one 3D mesh for the at least one virtual cuboid object.

### Claim 15 (depends on 14)

The device of claim 14, wherein the MPEG_extension includes data defining at least one of boundaries of the at least one 3D mesh for the at least one virtual cuboid object or a material for the 3D mesh.

### Claim 16 (depends on 11)

The device of claim 11, wherein to receive the first object information, the presentation engine is configured to receive data including one or more of:
boundaries data representing one or more collision boundaries of the at least one virtual cuboid object;
static data representing whether the at least one virtual solid object is affected by collisions; or
material data representing how colliding objects interact with the at least one virtual cuboid object.

### Claim 17 (depends on 11)

The device of claim 11, wherein the at least one virtual cuboid object comprises one of a virtual wall, a virtual chair, or a virtual table.

### Claim 18 (depends on 11)

The device of claim 11, wherein the streamed media data comprises glTF 2.0 media data.

### Claim 19 (depends on 11)

The device of claim 11, wherein to receive the streamed media data, the presentation engine is configured to request the streamed media data from a retrieval unit via an application programming interface (API).

### Claim 20 (depends on 11)

The device of claim 11, wherein the extension data structure is included in an MPEG scene description.

### Claim 21

A computer-readable storage medium having stored thereon instructions that, when executed, cause a processor to:
receive streamed media data representing a virtual three-dimensional (3D) scene;
receive an extension data structure associated with the virtual 3-D scene, the extension data structure including:
first object information comprising a vertex coordinate indicative of at least one virtual cuboid object in the virtual 3D scene, and
animation information indicative of an animation triggered based on interaction of a virtual camera with the at least one virtual cuboid object in the virtual 3D scene;
receive camera movement data from a user requesting that the virtual camera move through the at least one virtual cuboid object; and
using the first object information, update a location of the virtual camera to ensure the virtual camera remains outside of the at least one virtual cuboid object in response to the camera movement data.

### Claim 22 (depends on 21)

The computer-readable medium of claim 21, wherein the instructions that cause the processor to update the location of the virtual camera comprise instructions that cause the processor to prevent the virtual camera from passing through the at least virtual cuboid object.

### Claim 23 (depends on 21)

The computer-readable medium of claim 21, wherein the instructions that cause the processor to receive the extension data structure comprise instructions that cause the processor to receive an MPEG_extension.

### Claim 24 (depends on 22)

The computer-readable medium of claim 22, wherein the MPEG_extension further includes data defining at least one 3D mesh for the at least one virtual cuboid object.

### Claim 25 (depends on 23)

The computer-readable medium of claim 23, wherein the MPEG_mesh_collision extension includes data defining at least one of boundaries of a 3D mesh for the at least one virtual solid object, a material for the 3D mesh, or animations to be presented in response to the virtual camera contacting the 3D mesh.

### Claim 26 (depends on 21)

The computer-readable medium of claim 21, wherein the instructions that cause the processor to receive the first object information comprise instructions that cause the processor to receive data including one or more of:
boundaries data representing one or more collision boundaries of the at least one virtual cuboid object;
static data representing whether the at least one virtual cuboid object is affected by collisions; or
material data representing how colliding objects interact with the at least one virtual cuboid object.

### Claim 27 (depends on 21)

The computer-readable medium of claim 21, wherein the at least one virtual cuboid object comprises one of a virtual wall, a virtual chair, or a virtual table.

### Claim 28 (depends on 21)

The computer-readable medium of claim 21, wherein the streamed media data comprises glTF 2.0 media data.

### Claim 29 (depends on 21)

The computer-readable medium of claim 21, wherein the instructions that cause the processor to receive the streamed media data comprise instructions that cause the processor to request the streamed media data from a retrieval unit via an application programming interface (API).

### Claim 30 (depends on 21)

The computer-readable medium of claim 21, wherein the extension data structure is included in an MPEG scene description.

### Claim 31

A device for retrieving media data, the device comprising:
means for receiving streamed media data representing a virtual three-dimensional (3D) scene;
means for receiving an extension data structure associated with the virtual 3-D scene, the extension data structure including:
first object information comprising a vertex coordinate indicative of at least one virtual cuboid object in the virtual 3D scene, and
animation information indicative of an animation triggered based on interaction of a virtual camera with the at least one virtual cuboid object in the virtual 3D scene;
means for receiving camera movement data from a user requesting that the virtual camera move through the at least one virtual sew cuboid object; and
means for updating a location of the virtual camera to ensure the virtual camera remains outside of the at least one virtual cuboid object in response to the camera movement data using the first object information.