# US 10,120,187 B2 · Sub-frame scanout for latency reduction in virtual reality applications

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,120,187 B2
- Document ID: US10120187B2
- Status: Granted
- Kind: B2
- Application number: 15/047,568
- Filed: 2016-02-18
- Published: 2017-08-24
- Granted: 2018-11-06
- Earliest claimed: 2016-02-18
- Assignees: NVIDIA Corporation
- Applicants: NVIDIA Corporation
- Inventors: Craig Michael Wittenbrink; Ziyad Sami Hakura
- Web page: https://patentgrep.com/patent/US10120187B2

## CPC

- G02B 27/01: Head-up displays
- G02B 27/0093: {with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking}
- G02B 27/017: {Head mounted}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/14: Digital output to display device {; Cooperation and interconnection of the display device with other functional units}
- G06F 3/147: using display panels
- G06F 9/451: Execution arrangements for user interfaces
- G06T 15/00: Three-dimensional [3D] image rendering
- G09G 5/00: Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators (image data processing or generation, in general G06T)
- G09G 3/003: {to produce spatial visual effects}
- G09G 5/18: Timing circuits for raster scan displays (specially adapted for television H04N {; synchronisation between the display unit and other display units, videodisc player G09G5/12})
- G09G 5/363: {Graphics controllers}
- G09G 2320/0252: Improving the response speed
- G09G 2340/02: Handling of images in compressed format, e.g. JPEG, MPEG
- G09G 2340/14: Solving problems related to the presentation of information to be displayed
- G09G 2350/00: Solving problems of bandwidth in display systems
- G09G 2352/00: Parallel handling of streams of display data
- G09G 2360/06: Use of more than one graphics processor to process data before displaying to one or more screens
- G09G 2360/08: Power processing, i.e. workload management for processors involved in display operations, such as CPUs or GPUs
- G09G 2360/121: using a cache memory
- G09G 2370/022: Centralised management of display operation, e.g. in a server instead of locally
- G09G 2370/16: Use of wireless transmission of display information
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00)

## Abstract

A system, computer readable medium, and method for sub-frame scan-out are disclosed. The method includes the steps of dividing a frame into a plurality of slices. For each slice in the plurality of slices, the steps further include sampling a sensor associated with a head mounted display to generate sample data corresponding to the slice; adjusting one or more parameters associated with rendering operations for the slice based on the sample data; and rendering primitive data associated with a model according to the rendering operations to generate image data for the slice. Each slice is a portion of the frame and corresponds to different sample data from the sensor. Thus, adjusting of the parameters is different for each slice of the frame.

## Claims

### Claim 1

A method, comprising:
dividing a frame into a plurality of slices, wherein each slice is a portion of the frame; and
for each slice in the plurality of slices:
sampling a sensor associated with a head mounted display to generate sample data corresponding to the slice,
adjusting a viewport associated with rendering operations for the slice based on the sample data,
rendering primitives associated with a model according to the viewport and rendering operations to generate image data for the slice,
wherein the sampling, adjusting, and rendering is performed for each slice in the plurality of slices based on different sample data from the sensor for each slice in the frame, and the viewport changes for at least one slice of the plurality of slices.

### Claim 2 (depends on 1)

The method of claim 1, wherein the sensor comprises a multi-axis accelerometer.

### Claim 3 (depends on 1)

The method of claim 1, wherein the sensor comprises one or more of a gyroscope, an accelerometer, and a magnetometer.

### Claim 4 (depends on 1)

The method of claim 1, wherein the head mounted display comprises a first LCD display for a left eye of a viewer and a second LCD display for a right eye of a viewer.

### Claim 5 (depends on 1)

The method of claim 1, further comprising displaying the image data for each slice on at least one display device in the head mounted display.

### Claim 6 (depends on 5)

The method of claim 5, wherein the image data for a first slice is displayed on the at least one display device in the head mounted display substantially simultaneously with rendering the primitive data for a second slice on a graphics processing unit.

### Claim 7 (depends on 6)

The method of claim 6, wherein sample data for a third slice is read by a processor substantially simultaneously with rendering the primitive data for the second slice on the graphics processing unit and displaying the image data for the first slice on the at least one display device in the head mounted display.

### Claim 8 (depends on 1)

The method of claim 1, wherein the head mounted display includes a system on a chip (SoC) that includes at least one CPU core and a plurality of GPU cores.

### Claim 9 (depends on 1)

The method of claim 1, wherein the head mounted display is communicatively coupled to a device that includes a processor and a graphics processing unit, and wherein the image data is rendered on the device and transmitted to the head mounted display via a communications interface.

### Claim 10 (depends on 9)

The method of claim 9, wherein the communications interface is a wireless interface.

### Claim 11

A non-transitory, computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform steps comprising:
dividing a frame into a plurality of slices, wherein each slice is a portion of the frame; and
for each slice in the plurality of slices:
sampling a sensor associated with a head mounted display to generate sample data corresponding to the slice,
adjusting a viewport associated with rendering operations for the slice based on the sample data,
rendering primitives associated with a model according to the viewport and rendering operations to generate image data for the slice,
wherein the sampling, adjusting, and rendering is performed for each slice in the plurality of slices based on different sample data from the sensor for each slice in the frame, and the viewport changes for at least one slice of the plurality of slices.

### Claim 12 (depends on 11)

The computer-readable storage medium of claim 11, wherein the head mounted display comprises a first LCD display for a left eye of a viewer and a second LCD display for a right eye of a viewer.

### Claim 13 (depends on 11)

The computer-readable storage medium of claim 11, further comprising displaying the image data for each slice on at least one display device in the head mounted display.

### Claim 14 (depends on 13)

The computer-readable storage medium of claim 13, wherein the image data for a first slice is displayed on the at least one display device in the head mounted display substantially simultaneously with rendering the primitive data for a second slice on a graphics processing unit.

### Claim 15 (depends on 14)

The computer-readable storage medium of claim 14, wherein sample data for a third slice is read by a processor substantially simultaneously with rendering the primitive data for the second slice on the graphics processing unit and displaying the image data for the first slice on the at least one display device in the head mounted display.

### Claim 16 (depends on 11)

The computer-readable storage medium of claim 11, wherein the head mounted display includes a system on a chip (SoC) that includes at least one CPU core and a plurality of GPU cores.

### Claim 17

A system, comprising:
a head mounted display; and
at least one processor configured to:
divide a frame into a plurality of slices, wherein each slice is a portion of the frame, and
for each slice in the plurality of slices:
sample a sensor associated with the head mounted display to generate sample data corresponding to the slice,
adjust a viewport associated with rendering operations for the slice based on the sample data,
render primitives associated with a model according to the viewport and rendering operations to generate image data for the slice,
wherein the sampling, adjusting, and rendering is performed for each slice in the plurality of slices based on different sample data from the sensor for each slice in the frame, and the viewport changes for at least one slice of the plurality of slices.

### Claim 18 (depends on 17)

The system of claim 17, wherein the at least one processor comprises a system on a chip (SoC) that includes at least one CPU core and a plurality of GPU cores, and wherein the at least one CPU core is configured to adjust the viewport associated with rendering operations for the slice and the plurality of GPU cores are configured to render primitive data associated with the model to generate image data for the slice.

### Claim 19 (depends on 17)

The system of claim 17, wherein the head mounted display comprises a first LCD display for a left eye of a viewer and a second LCD display for a right eye of a viewer.

### Claim 20 (depends on 17)

The system of claim 17, wherein the image data for a first slice is displayed on the head mounted display substantially simultaneously with rendering the primitive data for a second slice.