# US 11,195,250 B2 · Prediction system for texture streaming

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,195,250 B2
- Document ID: US11195250B2
- Status: Granted
- Kind: B2
- Application number: 16/945,522
- Filed: 2020-07-31
- Published: 2020-12-31
- Granted: 2021-12-07
- Earliest claimed: 2014-11-03
- Assignees: Graphine NV
- Applicants: Graphine NV
- Inventors: Bart Pieters; Charles-Frederik Hollemeersch; Aljosha Demeulemeester
- Web page: https://patentgrep.com/patent/US11195250B2

## CPC

- G06T 1/60: Memory management
- G06T 15/04: Texture mapping
- G06T 19/00: Manipulating three-dimensional [3D] models or images for computer graphics
- H04L 65/612: for unicast
- H04L 65/70: Media network packetisation
- H04L 65/80: Responding to QoS
- G06T 2210/08: Bandwidth reduction
- G06T 2210/36: Level of detail
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

The present invention relates to a prediction system for determining a set of subregions to be used for rendering a virtual world of a computer graphics application, said subregions belonging to streamable objects to be used for rendering said virtual world, said streamable objects each comprising a plurality of subregions. The prediction system comprises a plurality of predictor units arranged for receiving from a computer graphics application information on the virtual world and each arranged for obtaining a predicted set of subregions for rendering a virtual world using streamable objects, each predicted set being obtained by applying a different prediction scheme, a streaming manager arranged for receiving the predicted sets of subregions, for deriving from the predicted sets a working set of subregions to be used for rendering and for outputting, based on the working set of subregions, steering instructions concerning the set of subregions to be actually used.

## Claims

### Claim 1

A system comprising:
one or more computer processors;
one or more computer memories; and
a set of instructions incorporated into the one or more computer memories, the set of instructions configuring the one or more computer processors to perform operations for facilitating rendering of a virtual world of a computer graphics application, the operations comprising:
receiving data describing the virtual world from the computer graphics application;
analyzing the received data using a plurality of predictors, the analyzing including applying a different prediction scheme according to each predictor of the plurality of predictors to identify a predicted set of subregions belonging to streamable objects to be used for rendering the virtual world;
combining the predicted sets of subregions into a working set of subregions to be used for the rendering, the combining based on an analysis of output from each predictor;
determining priorities of steering instructions for loading subregions of the working set into a streaming cache memory to be used for the rendering, the determining of the priorities based on an analysis of output from each predictor; and
outputting the steering instructions to a streaming engine to perform the loading of the subregions in order of the priorities.

### Claim 2 (depends on 1)

The system as in claim 1, wherein at least one streamable object is represented by subregions corresponding to a plurality of different level-of-detail versions of the at least one streamable object.

### Claim 3 (depends on 1)

The system as in claim 1, the operations further including applying a timing scheme for the application of a prediction scheme by one or more predictors of the plurality of predictors.

### Claim 4 (depends on 3)

The system as in claim 3, wherein the timing scheme is determined at a run time of the application.

### Claim 5 (depends on 3)

The system as in claim 3, wherein the timing scheme is determined in accordance with available system resources.

### Claim 6 (depends on 3)

The system as in claim 3, wherein the timing scheme is determined based on a computational complexity of a prediction scheme applied by the one or more predictors.

### Claim 7 (depends on 3)

The system as in claim 3, wherein the timing scheme is determined based on an accuracy of results of the predicted sets of subregions and/or a delay at which the predicted sets become available.

### Claim 8 (depends on 1)

The system as in claim 1, wherein each prediction scheme has a different measure of accuracy of the output.

### Claim 9 (depends on 1)

The system as in claim 1, wherein one of the predicted sets is generated by determining which streamable objects or which subregions are within a predetermined virtual distance from a position in the virtual world.

### Claim 10 (depends on 1)

The system as in claim 1, wherein at least one of the prediction schemes comprises rendering a scene from a virtual camera position in the virtual world and identifying a set of unique subregion identifiers associated with streamable objects visible by a virtual camera at the virtual camera position, and outputting the set of unique subregion identifiers.

### Claim 11 (depends on 1)

The system as in claim 1, wherein at least one of the prediction schemes is configured for performing a first prediction associated with a main virtual camera position in the virtual world, and a second prediction associated with a second virtual camera position, the second virtual camera position being associated with a future main virtual camera position.

### Claim 12 (depends on 1)

The system as in claim 1, wherein at least one of the prediction schemes is configured for performing a prediction whereby streamable objects in the virtual world are rendered from a point of view with no occluding objects or wherein occluding objects are rendered with a level of transparency.

### Claim 13 (depends on 1)

The system as in claim 1, the operations further including assigning a weight to the various subregions of the predicted sets, the weight determining a priority for inclusion in the working set and a priority within the steering instructions.

### Claim 14 (depends on 13)

The system as in claim 13, wherein the weight takes into account a number of occurrences of a subregion in the plurality of predicted sets.

### Claim 15 (depends on 1)

The system as in claim 1, the operations further including selecting memory slots to be reused and forwarding an indication of the selected memory slots in the instructions.

### Claim 16 (depends on 1)

The system as in claim 1, the operations further including:
modifying an output of one or more predictors using the output of at least one other predictor; and
performing at least one of adjusting the combining of the predicted sets of subregions based on an analysis of the modified output and adjusting the determining of priorities based on an analysis of the modified output.

### Claim 17 (depends on 1)

The system of claim 1, wherein the combining of the predicted sets of subregions into the working set of subregions to be used for the rendering is based on an analysis of a measure of output accuracy from each predictor.

### Claim 18 (depends on 1)

The system of claim 1, wherein the determining of the priorities of steering instructions is based on an analysis of a measure of output accuracy from each predictor.

### Claim 19

A non-transitory computer-readable storage medium storing a set of instructions that, when executed by one or more processors, causes the one or more processors to perform operations for facilitating rendering of a virtual world of a computer graphics application, the operations comprising:
receiving data describing the virtual world from the computer graphics application;
analyzing the received data using a plurality of predictors, the analyzing including applying a different prediction scheme according to each predictor of the plurality of predictors to identify a predicted set of subregions belonging to streamable objects to be used for rendering the virtual world;
combining the predicted sets of subregions into a working set of subregions to be used for the rendering, the combining based on an analysis of output from each predictor;
determining priorities of steering instructions for loading subregions of the working set into a streaming cache memory to be used for the rendering, the determining of the priorities based on an analysis of output from each predictor; and
outputting the steering instructions to a streaming engine to perform the loading of the subregions in order of the priorities.

### Claim 20

A method comprising:
performing, using one or more computer processors, operations for facilitating rendering of a virtual world of a computer graphics application, the operations comprising:
receiving data describing the virtual world from the computer graphics application;
analyzing the received data using a plurality of predictors, the analyzing including applying a different prediction scheme according to each predictor of the plurality of predictors to identify a predicted set of subregions belonging to streamable objects to be used for rendering the virtual world;
combining the predicted sets of subregions into a working set of subregions to be used for the rendering, the combining based on an analysis of output from each predictor;
determining priorities of steering instructions for loading subregions of the working set into a streaming cache memory to be used for the rendering, the determining of the priorities based on an analysis of output from each predictor; and
outputting the steering instructions to a streaming engine to perform the loading of the subregions in order of the priorities.
