# US 10,313,679 B2 · Systems and methods for encoder-guided adaptive-quality rendering

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,313,679 B2
- Document ID: US10313679B2
- Status: Granted
- Kind: B2
- Application number: 15/959,069
- Filed: 2018-04-20
- Published: 2018-10-25
- Granted: 2019-06-04
- Earliest claimed: 2017-04-21
- Assignees: ZeniMaz Media Inc.
- Applicants: ZeniMax Media Inc.
- Inventors: Michael Kopietz
- Web page: https://patentgrep.com/patent/US10313679B2

## CPC

- H04N 19/172: the region being a picture, frame or field
- A63F 13/30: Interconnection arrangements between game servers and game devices; Interconnection arrangements between game devices; Interconnection arrangements between game servers
- A63F 13/355: Performing operations on behalf of clients with restricted processing capabilities, e.g. servers transform changing game scene into an encoded video stream for transmitting to a mobile phone or a thin client
- A63F 13/52: involving aspects of the displayed game scene
- G06T 15/005: {General purpose rendering architectures}
- H04N 19/124: Quantisation
- H04N 19/154: Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion (use of rate-distortion criteria H04N19/147)
- H04N 19/156: Availability of hardware or computational resources, e.g. encoding based on power-saving criteria
- H04N 19/192: the adaptation method, adaptation tool or adaptation type being iterative or recursive
- H04N 19/85: using pre-processing or post-processing specially adapted for video compression
- A63F 2300/538: for performing operations on behalf of the game client, e.g. rendering
- A63F 2300/66: for rendering three dimensional images
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Systems and methods for improving computer technology related to the rendering and encoding of images are disclosed, preferably for use in a video-game environment. In certain embodiments, a codec is used to encode one or more reference images for a partial range of encoder settings and a renderer is used to generate one or more rendering quality-settings profiles, generate one or more reference images, calculate perceived qualities for each of the one or more reference images, re-render the one or more reference images for each of the one or more rendering quality-setting profiles, and calculate perceived qualities for each of the one or more re-rendered reference images. The renderer compares the perceived qualities of the reference images to the perceived qualities of the re-rendered images and matches them. Those matches result in an association of one or more encoder settings with their matching rendering quality-settings profiles into a look-up table. The lookup table is used to generate a rendered image at a substantially identical perceived quality to an encoded frame during gameplay.

## Claims

### Claim 1

A computer-implemented method for rendering, comprising the steps of:
generating one or more rendering quality-settings profiles;
generating one or more reference images;
encoding the one or more reference images for a partial range of encoder settings;
calculating a first perceived quality for each of the one or more reference images;
re-rendering the one or more reference images for each of the one or more rendering quality-setting profiles;
calculating one or more second perceived qualities for each of the one or more re-rendered reference images;
comparing the one or more first perceived qualities to the one or more second perceived qualities, wherein a match between one or more first perceived qualities and the one or more second perceived qualities results in an association of one or more encoder settings with their matching rendering quality-settings profiles in one or more look-up tables; and
generating a rendered image at a substantially identical perceived quality to an encoded frame on the basis of the look-up table.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, further comprising the step of calculating a computational cost for each of the re-rendered reference images.

### Claim 3 (depends on 2)

The computer-implemented method of claim 2, further comprising the step of, for the reference images and the re-rendered reference images, applying the computational cost of each of the re-rendered reference images to tiebreak multiple matches of the one or more first perceived qualities to the one or more second perceived qualities.

### Claim 4 (depends on 2)

The computer-implemented method of claim 2, wherein the computational cost is measured in rendering time or clock-cycles.

### Claim 5 (depends on 1)

The computer-implemented method of claim 1, wherein an algorithmic image quality assessment is computed and then used to calculate the first and second perceived qualities.

### Claim 6 (depends on 1)

The computer-implemented method of claim 1, wherein the association of one or more encoder settings with their matching rendering quality-settings profiles is performed by applying a decision tree to programmatically narrow down possibility space.

### Claim 7 (depends on 1)

The computer-implemented method of claim 1, wherein a structural similarity index (SSIM) is used to measure the first perceived qualities and the second perceived qualities.

### Claim 8 (depends on 1)

The computer-implemented method of claim 1, wherein the one or more look-up tables are generated using a plurality of integer values or non-integer partial ranges of quantization parameters.

### Claim 9 (depends on 8)

The computer-implemented method of claim 8, wherein the integer values of quantization parameters have a range of 0 to 51.

### Claim 10 (depends on 1)

The computer-implemented method of claim 1, wherein the quality-settings profile is a list of values for each available rendering quality setting.

### Claim 11

A system for rendering comprising:
a codec that encodes one or more reference images for a partial range of encoder settings; and
a renderer; wherein the renderer:
generates one or more rendering quality-settings profiles;
generates the one or more reference images;
calculates a first perceived quality for each of the one or more reference images;
re-renders the one or more reference images for each of the one or more rendering quality-setting profiles;
calculates one or more second perceived qualities for each of the one or more re-rendered reference images;
compares the one or more first perceived qualities to the one or more second perceived qualities, wherein a match between one or more first perceived qualities and the one or more second perceived qualities results in an association of one or more encoder settings with their matching rendering quality-settings profiles in one or more look-up tables; and
generates a rendered image at a substantially identical perceived quality to an encoded frame on the basis of the look-up table.

### Claim 12 (depends on 11)

The system of claim 11, further comprising the step of calculating a computational cost for each of the re-rendered reference images.

### Claim 13 (depends on 12)

The system of claim 12, further comprising step of, for the reference images and the re-rendered reference images, applying the computational cost of each of the re-rendered reference images to tiebreak multiple matches of the one or more first perceived qualities to the one or more second perceived qualities.

### Claim 14 (depends on 12)

The system of claim 12, wherein the computational cost is measured in rendering time or clock-cycles.

### Claim 15 (depends on 11)

The system of claim 11, wherein an algorithmic image quality assessment is computed and then a is used to calculate the first and second perceived qualities.

### Claim 16 (depends on 11)

The system of claim 11, wherein the association of one or more encoder settings with their matching rendering quality-settings profiles is performed by applying a decision tree to programmatically narrow down possibility space.

### Claim 17 (depends on 11)

The system of claim 11, wherein a structural similarity index (SSIM) is used to measure the first perceived qualities and the second perceived qualities.

### Claim 18 (depends on 11)

The system of claim 11, wherein the one or more look-up tables are generated using a plurality of integer values or non-integer partial ranges of quantization parameters.

### Claim 19 (depends on 18)

The system of claim 18, wherein the integer values of quantization parameters have a range of 0 to 51.

### Claim 20 (depends on 11)

The system of claim 11, wherein the quality-settings profile is a list of values for each available rendering quality setting.