# US 2021/0343261 A1 · Systems and methods for foveated rendering

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0343261 A1
- Document ID: US20210343261A1
- Status: Published application
- Kind: A1
- Application number: 17/377,288
- Filed: 2021-07-15
- Published: 2021-11-04
- Earliest claimed: 2019-08-13
- Assignees: Facebook Technologies, LLC
- Applicants: Facebook Technologies, LLC
- Inventors: Larry Seiler
- Web page: https://patentgrep.com/patent/US20210343261A1

## CPC

- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G09G 5/37: Details of the operation on graphic patterns (G09G5/38 takes precedence)
- G06F 3/012: {Head tracking input arrangements}
- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06T 1/20: Processor architectures; Processor configuration, e.g. pipelining
- G06T 5/73: Deblurring; Sharpening
- G06T 15/06: Ray-tracing
- G09G 5/02: characterised by the way in which colour is displayed {(details of colour display specific for CRTs G09G1/28; specific for flat matrix panels other than liquid crystal displays G09G3/2003; specific for liquid crystal displays G09G3/3607)}
- H04N 13/106: Processing image signals (for multi-view video sequence encoding H04N19/597)
- H04N 13/344: with head-mounted left-right displays
- H04N 13/366: using viewer tracking
- H04N 13/398: Synchronisation thereof; Control thereof
- G06T 2207/20192: Edge enhancement; Edge preservation
- G09G 2320/0261: in the context of movement of objects on the screen or movement of the observer relative to the screen
- G09G 2320/0686: with two or more screen areas displaying information with different brightness or colours
- G09G 2340/0435: Change or adaptation of the frame rate of the video stream
- G09G 2354/00: Aspects of interface with display user
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

In one embodiment, a computing system may determine a focus point of a viewer based on received sensor data. The system may determine, for a current frame, a first viewing region encompassing a focus point of the viewer and a second view region excluding the first viewing region. The system may determine, for the current frame, color values for the first viewing region using respective first sampling resolutions, and color values for the second viewing region using respective second sampling resolutions. At least one second sampling resolution may be lower than a corresponding first sampling resolution associated with a same color channel. At least two of the second sampling resolutions for the color channels of the second viewing region may be different from each other. The system may output the color values for the first viewing region and the second viewing region of the current frame for display.

## Claims

### Claim at position 1

1-20. (canceled)

### Claim 21

A method comprising, by a computing system:
determining a focus point of a viewer based on sensor data received by the computing system;
determining, for a current frame, a first viewing region encompassing a focus point of the viewer and a second view region excluding the first viewing region;
determining, for the current frame, color values for the first viewing region using respective first sampling resolutions for a plurality of color channels, and color values for the second viewing region using respective second sampling resolutions for the plurality of color channels, wherein at least one second sampling resolution is lower than a corresponding first sampling resolution associated with a same color channel, and wherein at least two of the second sampling resolutions for the plurality of color channels of the second viewing region are different from each other; and
outputting the color values for the first viewing region and the second viewing region of the current frame for display.

### Claim 22 (depends on 21)

The method of claim 21, wherein the color values corresponding to the first and second view regions are determined based on a plurality of tile-surface pairs, and wherein the plurality of title-surface pairs are determined by:
casting a plurality of rays to a plurality of surfaces for generating the current frame;
determining intersections of the plurality of rays and the plurality of surfaces; and
determining the plurality of tile-surface pairs based on the intersections of the plurality of rays and the plurality of surfaces.

### Claim 23 (depends on 22)

The method of claim 22, wherein the color values corresponding to the second sample view region are determined using a smaller amount of computation than the color values corresponding to the first viewing region.

### Claim 24 (depends on 22)

The method of claim 22, wherein the color values corresponding to the second viewing region are determined based on a smaller number casted rays than the color values corresponding to the first viewing region.

### Claim 25 (depends on 21)

The method of claim 21, wherein color values of a first color channel of the first viewing region and color values of the first color channel of the second viewing region are determined using a same sampling resolution.

### Claim 26 (depends on 21)

The method of claim 21, wherein color values of a second color channel of the first viewing region and color values of the second color channel of the second viewing region are determined using two sampling resolutions different from each other, and wherein the two sample resolutions have a relationship of powers of two.

### Claim 27 (depends on 21)

The method of claim 21, further comprising:
determining a n value based on a first sampling resolution associated with a first color channel of the first viewing region; and
determining a color value for each n×n pixel array of the first color channel of the first viewing region, wherein each pixel in the n×n pixel array shares the color value.

### Claim 28 (depends on 27)

The method of claim 27, further comprising:
determining a color value for each pixel within the n×n pixel array based on a replication process performed by a second computing system on a display.

### Claim 29 (depends on 27)

The method of claim 27, further comprising:
determining a color value for each pixel within the n×n pixel array by interpolating the color values associated with the n×n pixel array and one or more adjacent n×n pixel arrays.

### Claim 30 (depends on 21)

The method of claim 21, further comprising:
determining a third viewing region surrounding the second viewing region, wherein color values of the plurality of color channels corresponding to the third viewing region are determined using a same third sampling resolution.

### Claim 31 (depends on 21)

The method of claim 21, further comprising, prior to determining the color values of the first viewing regions and the second viewing regions:
pre-processing a source data with successively lower resolutions for generating the current frame; and
accessing the source data with the successively lower resolutions while generating the current frame.

### Claim 32 (depends on 21)

The method of claim 21, further comprising:
applying a sharpness filter to a plurality of pixels of the current frame in the second viewing region, wherein the current frame preserves a contrast level on one or more edges associated with one or more objects in the second viewing region.

### Claim 33 (depends on 21)

The method of claim 21, further comprising:
applying a sharpness filter to a plurality of pixels of the current frame in the second viewing region, wherein the current frame preserves an average brightness in the second viewing region.

### Claim 34 (depends on 21)

The method of claim 21, wherein the plurality of color channels comprise a red color channel, a blue color channel, and a green color channels, wherein the red color channel and the blue color channel of the second viewing region have a same sampling resolution, and wherein the green color channel has a higher sampling resolution than the red color channel and the blue color channel.

### Claim 35

One or more computer-readable non-transitory storage media embodying software that is operable when executed to:
determine a focus point of a viewer based on sensor data received by the computing system;
determine, for a current frame, a first viewing region encompassing a focus point of the viewer and a second view region excluding the first viewing region;
determine, for the current frame, color values for the first viewing region using respective first sampling resolutions for a plurality of color channels, and color values for the second viewing region using respective second sampling resolutions for the plurality of color channels, wherein at least one second sampling resolution is lower than a corresponding first sampling resolution associated with a same color channel, and wherein at least two of the second sampling resolutions for the plurality of color channels of the second viewing region are different from each other; and
output the color values for the first viewing region and the second viewing region of the current frame for display.

### Claim 36 (depends on 35)

The media of claim 35, wherein the color values corresponding to the first and second view regions are determined based on a plurality of tile-surface pairs, and wherein the plurality of title-surface pairs are determined by:
casting a plurality of rays to a plurality of surfaces for generating the current frame;
determining intersections of the plurality of rays and the plurality of surfaces; and
determining the plurality of tile-surface pairs based on the intersections of the plurality of rays and the plurality of surfaces.

### Claim 37 (depends on 36)

The media of claim 36, wherein the color values corresponding to the second sample view region are determined using a smaller amount of computation than the color values corresponding to the first viewing region.

### Claim 38

A system comprising:
one or more processors, and
one or more computer-readable non-transitory storage media coupled to one or more of the processors and comprising instructions operable when executed by one or more of the processors to cause the system to:
determine a focus point of a viewer based on sensor data received by the computing system;
determine, for a current frame, a first viewing region encompassing a focus point of the viewer and a second view region excluding the first viewing region;
determine, for the current frame, color values for the first viewing region using respective first sampling resolutions for a plurality of color channels, and color values for the second viewing region using respective second sampling resolutions for the plurality of color channels, wherein at least one second sampling resolution is lower than a corresponding first sampling resolution associated with a same color channel, and wherein at least two of the second sampling resolutions for the plurality of color channels of the second viewing region are different from each other; and
output the color values for the first viewing region and the second viewing region of the current frame for display.

### Claim 39 (depends on 38)

The system of claim 38, wherein the color values corresponding to the first and second view regions are determined based on a plurality of tile-surface pairs, and wherein the plurality of title-surface pairs are determined by:
casting a plurality of rays to a plurality of surfaces for generating the current frame;
determining intersections of the plurality of rays and the plurality of surfaces; and
determining the plurality of tile-surface pairs based on the intersections of the plurality of rays and the plurality of surfaces.

### Claim 40 (depends on 39)

The system of claim 39, wherein the color values corresponding to the second sample view region are determined using a smaller amount of computation than the color values corresponding to the first viewing region.
