# US 2020/0402294 A1 · 3d structure engine-based computation platform

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0402294 A1
- Document ID: US20200402294A1
- Status: Published application
- Kind: A1
- Application number: 16/904,371
- Filed: 2020-06-17
- Published: 2020-12-24
- Earliest claimed: 2019-06-18
- Assignees: TMRW Foundation IP & Holding S. À R.L.
- Applicants: TMRW Foundation IP & Holding S. À R.L.
- Inventors: Cevat Yerli
- Web page: https://patentgrep.com/patent/US20200402294A1

## CPC

- G06F 9/46: Multiprogramming arrangements
- G06T 15/205: {Image-based rendering}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06T 7/536: from perspective effects, e.g. by using vanishing points
- G06T 15/005: {General purpose rendering architectures}
- G06T 15/08: Volume rendering
- G06T 17/10: Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes
- G06T 2215/16: Using real world measurements to influence rendering
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A system and method enabling per-user-optimized computing, rendering, and provisioning within virtual worlds. The system comprises a server including memory and at least one processor, the memory storing a persistent virtual world system comprising a data structure in which at least one virtual replica of at least one corresponding real object is represented, and a computing optimization platform configured to store and provide rules for optimizing the computing, rendering and data provisioning to users via user devices. A plurality of connected devices connected to the server via a network provide multi-source data, user input, or combinations thereof, to the persistent virtual world system, updating the virtual replicas. The server retrieves user location, viewing position and orientation from the one or more user devices to determine a user interaction radius, thereby optimizing via the computing optimization platform the relevant computing, rendering and provisioning for the one or more user devices.

## Claims

### Claim 1

A system enabling computing, rendering, and provisioning within virtual worlds, the system comprising:
a server system comprising one or more server computers, the one or more server computers comprising memory and at least one processor, the server system storing a persistent virtual world system comprising a data structure in which at least one virtual replica of at least one corresponding real object is represented, and a computing optimization platform configured to store and provide rules for optimizing computing, rendering and data provisioning for the persistent virtual world system to users via one or more user devices;
wherein the server system retrieves user location, viewing position and orientation from the one or more user devices to determine a user interaction radius, thereby optimizing via the computing optimization platform the relevant computing, rendering and provisioning for the one or more user devices, wherein the optimizations are performed on a user interaction radius basis.

### Claim 2 (depends on 1)

The system of claim 1, further comprising a plurality of connected devices configured to capture multi-source data that serves to update the at least one virtual replica, and one or more fog servers comprising memory and at least one processor, the fog servers being located in areas proximate to the connected devices and configured to assist the server system and connected devices in the processing of the persistent virtual world system.

### Claim 3 (depends on 2)

The system of claim 2, further comprising a virtual layer abstracting functions between the connected devices, the server system and the one or more fog servers, the virtual layer further comprising a distribution platform configured to distribute the relevant computing, rendering and provisioning across edge and cloud based on the rules provided by the computing optimization platform.

### Claim 4 (depends on 3)

The system of claim 3, wherein the at least one distribution platform comprises one or more virtual cells linked to a plurality of physical network resource sources, and wherein two or more virtual cells may be used in combination in order to dynamically allocate resources and engine tasks to the one or more user devices as part of the provisioning to the user devices.

### Claim 5 (depends on 1)

The system of claim 1, wherein the server system further stores a replica editor configured to model and edit the one or more virtual replicas; data from the real world captured by sensing mechanisms on one or more connected devices; applications that are positioned within the persistent virtual world system; and an operating system configured to manage computer hardware and software resources for performing per-user-optimized computing, rendering and provisioning.

### Claim 6 (depends on 1)

The system of claim 1, wherein optimizations performed by the computing optimization platform are done on application data of applications positioned in the persistent virtual world system based on interactions with users, virtual replicas, other applications or virtual objects of the persistent virtual world system.

### Claim 7 (depends on 1)

The system of claim 1, wherein the computing optimization platform comprises computer programs enabling optimizations incorporating contextual data, 3D data structure, scene graphs, and preparation for rendering stages, and wherein the user interaction radius is comprised within the contextual data.

### Claim 8 (depends on 7)

The system of claim 7, wherein the 3D data structure comprises one or more octrees, quadtrees, BSP trees, sparse voxel octrees, 3D arrays, kD trees, point clouds, wire-frames, boundary representations (B-Rep), constructive solid geometry trees (CSG Trees), bintrees, or hexagonal structures, or combinations thereof.

### Claim 9 (depends on 7)

The system of claim 7, wherein the contextual data comprises a micro-context and a macro-context, the micro-context comprising user-related data and user interaction radius data, and the macro-context comprising network equipment, real world objects, virtual objects, number of users, or resource management, or a combination thereof.

### Claim 10 (depends on 9)

The system of claim 9, wherein the user-related data comprises user location data, user viewing position and orientation, user social data, user preferences, user service context, or quality of service specific to the user, or a combination thereof, and wherein the user interaction radius data comprises real world objects, virtual objects, or other users within the user interaction radius, or a combination thereof.

### Claim 11

A method enabling per-user-optimized computing of virtual worlds, the method comprising:
providing in memory of a server system comprising one or more server computers a persistent virtual world system comprising a data structure in which at least one virtual replica of at least one corresponding real object is represented, and a computing optimization platform configured to store and provide rules for optimizing computing, rendering and data provisioning for the persistent virtual world system to users via one or more user devices;
retrieving, by the server system, a user location, viewing position and orientation from the one or more user devices;
determining a user interaction radius around the one or more user devices; and
optimizing, via the computing optimization platform, the relevant computing, rendering and provisioning for the one or more user devices, wherein the optimizations are performed on a user interaction radius basis.

### Claim 12 (depends on 11)

The method of claim 11, further comprising providing one or more fog servers comprising memory and at least one processor, the fog servers being located in areas proximate to connected devices and configured to assist the server system and connected devices in the processing of the persistent virtual world system.

### Claim 13 (depends on 12)

The method of claim 12, further comprising providing a virtual layer abstracting functions between the connected devices, the server system and the one or more fog servers, the virtual layer further comprising a distribution platform configured to distribute the relevant computing, rendering and provisioning across edge and cloud based on the rules provided by the computing optimization platform.

### Claim 14 (depends on 13)

The method of claim 13, wherein the at least one distribution platform comprises one or more virtual cells linked to a plurality of physical network resource sources, and wherein two or more virtual cells may be used in combination in order to dynamically allocate resources and engine tasks to the one or more user devices as part of the provisioning to the user devices.

### Claim 15 (depends on 11)

The method of claim 11 further comprising providing in the server system a replica editor configured to model and edit the at least one virtual replica; data from the real world captured by sensing mechanisms on one or more connected devices; applications that are positioned within the persistent virtual world system; and an operating system configured to manage computer hardware and software resources for performing per-user-optimized computing, rendering and provisioning.

### Claim 16 (depends on 11)

The method of claim 11, wherein optimizations performed by the computing optimization platform are done on application data of applications positioned in the persistent virtual world system based on interactions with users, virtual replicas, other applications or virtual objects of the persistent virtual world system.

### Claim 17 (depends on 11)

The method of claim 11, wherein optimizing the relevant computing, rendering and provisioning for the one or more user devices by the computing optimization platform comprises:
performing optimizations incorporating contextual data, wherein the user interaction radius is comprised within the contextual data;
performing 3D data structure optimizations;
performing scene graphs optimizations; and
preparing for rendering stages.

### Claim 18 (depends on 17)

The method of claim 17, wherein the contextual data comprises a micro-context and macro-context, the micro-context comprising user-related data and user interaction radius data, and the macro-context comprising network equipment, real world objects, virtual objects, number of users, or resource management, or a combination thereof.

### Claim 19 (depends on 17)

The method of claim 17, wherein preparing for the rendering stages comprises performing depth-culling, frustum culling, high-level occlusion culling, or implementation level-of-detail (LOD) algorithms, or a combination thereof.

### Claim 20

One or more non-transitory computer-readable media having stored thereon instructions configured to, when executed by one or more computers, cause the one or more computers to:
provide in memory of a server system comprising one or more server computers a persistent virtual world system comprising a data structure in which at least one virtual replica of at least one corresponding real object is represented, and a computing optimization platform configured to store and provide rules for optimizing computing, rendering and data provisioning for the persistent virtual world system to users via one or more user devices;
retrieve, by the server system, a user location, viewing position and orientation from the one or more user devices;
determine a user interaction radius around the one or more user devices; and
optimize, via the computing optimization platform, the relevant computing, rendering and provisioning for the one or more user devices.
