# US 10,503,457 B2 · Method and apparatus for rendering perspective-correct images for a tilted multi-display environment

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,503,457 B2
- Document ID: US10503457B2
- Status: Granted
- Kind: B2
- Application number: 15/652,116
- Filed: 2017-07-17
- Published: 2018-11-08
- Granted: 2019-12-10
- Earliest claimed: 2017-05-05
- Assignees: NVIDIA Corporation
- Applicants: Nvidia Corporation
- Inventors: Rouslan Dimitrov; Yury Uralsky; Lars Nordskog; Dmitry Zhdan
- Web page: https://patentgrep.com/patent/US10503457B2

## CPC

- G06F 3/1446: {display composed of modules, e.g. video walls}
- G06F 3/1431: {using a single graphics controller}
- G06T 15/005: {General purpose rendering architectures}
- G06T 15/205: {Image-based rendering}
- G09G 5/14: Display of multiple viewports
- G09G 5/373: for modifying the size of the graphic pattern
- G09G 5/395: Arrangements specially adapted for transferring the contents of the bit-mapped memory to the screen (G09G5/399 takes precedence)
- G09G 2320/0693: Calibration of display systems
- G09G 2340/04: Changes in size, position or resolution of an image
- G09G 2340/0442: Handling or displaying different aspect ratios, or changing the aspect ratio
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00)

## Abstract

Techniques for rendering images on multiple tilted displays concurrently to mitigate perspective distortion are disclosed herein. According to one described approach, viewports are assigned to a center monitor and two peripheral monitors. Scene data for the viewports is calculated, and geometric primitives are generated for the viewports based on the scene data. Image transformation is performed based on a modified perspective value to modify geometry of the geometric primitives based on tilt angles of the displays, and the geometric primitives are rasterized using the modified geometry.

## Claims

### Claim 1

A method of generating a multi-screen rendering for a tilted multi-monitor display system, said method comprising:
creating a common viewport for a central monitor, a first peripheral monitor, and a second peripheral monitor, wherein the common viewport encapsulates final geometry to be displayed on the monitors, and wherein the first peripheral monitor and the second peripheral monitor are tilted at an angle relative to the center monitor;
calculating scene data based on the common viewport and a viewing position to generate geometric primitives for the center monitor, the first peripheral monitor, and the second peripheral monitor;
rasterizing the geometric primitives for the center monitor, the first peripheral monitor, and the second peripheral monitor to generate respective bitmaps for the center monitor, the first peripheral monitor, and the second peripheral monitor;
adjusting the respective bitmaps at the pixel level for the first peripheral monitor and the second peripheral monitor based on the angle relative to the center monitor; and
rendering perspective correct images on the center monitor, the first peripheral monitor, and the second peripheral monitor using the respective bitmaps.

### Claim 2 (depends on 1)

The method as described in claim 1 wherein the adjusting the respective bitmaps for the first peripheral monitor and the second peripheral monitor based on the angle relative to the center monitor comprises performing at least one of oversampling and resampling pixels of the respective bitmaps.

### Claim 3 (depends on 1)

The method as described in claim 1 further comprising obtaining a distance between the center monitor and a user and adjusting the respective bitmaps at the pixel level for the first peripheral monitor and the second peripheral monitor based on the distance.

### Claim 4 (depends on 1)

The method as described in claim 1, further comprising determining a portion of the bitmaps that will not be visible based on the common viewport, wherein the rendering perspective correct images comprises skipping the portion of the bitmaps that will not be visible.

### Claim 5 (depends on 1)

A method as described in claim 1 wherein said adjusting the respective bitmaps at the pixel level is performed by a plurality of triangle transformation engines, and wherein dedicated triangle transformation engines are associated with each of the center monitor, the first peripheral monitor, and the second peripheral monitor.

### Claim 6 (depends on 1)

A method as described in claim 1 wherein said adjusting the respective bitmaps comprises performing image-space processing using a virtual plane that is tilted relative to the angle of the peripheral displays.

### Claim 7 (depends on 6)

A method as described in claim 6 wherein said performing image-space processing comprises at least one of cropping and warping the bitmap.

### Claim 8

A method of generating a multi-screen rendering, said method comprising:
with respect to a three dimensional represented space comprising three dimensional defined objects therein, using a single pass of a geometry engine of a graphics processor to generate a projection, in 2 dimensions, of said space as viewed from a camera of a large common viewport, said large common viewport defined within a common plane that is perpendicular to a direction of view of said camera wherein said common plane is located a distance, D, from said camera and wherein said large common viewport encapsulates all geometry to be displayed on a first peripheral monitor, a second peripheral monitor, and a central monitor;
said geometry engine outputting triangles corresponding to said large common viewport to a rasterizer wherein said rasterizer generates a respective frame buffer bitmap for the said large common viewport;
said rasterizer outputting respective frame buffers corresponding to respective pixel transformation engines wherein each respective pixel transformation engine adjusts its respective frame buffer, in pixel space, based on: 1) a tilt angle of a respective monitor; 2) said distance D; and 3) a location in 3D space of said large common viewport to generate an output corrected frame buffer for the large common viewport that is perspective corrected for said respective monitor; and
rendering respective corrected frame buffers from said respective rasterizers onto said large common viewport.

### Claim 9 (depends on 8)

A method as described in claim 8 wherein said geometry engine outputs triangles having vertices in the format: (X, Y, Z and W).

### Claim 10 (depends on 8)

A method as described in claim 8 wherein said tilt angle is measured relative to said common plane.

### Claim 11 (depends on 8)

The method as described in claim 8 further comprising performing calibration to determine said D and any tilt angle of any peripheral monitor, wherein said performing calibration comprises:
a) projecting a geometric pattern onto said large common viewport;
b) receiving a user input to alter a correspondence to D to vary said D;
c) receive a user input to alter a correspondence to a tilt angle of the large common viewport to vary said tilt angle;
d) re-projecting an altered geometric pattern onto said large common viewport responsive to variations in said D and said tilt angle;
e) repeating said b)-d); and
f) capturing said D and said tilt angle responsive to a user calibration end input.

### Claim 12 (depends on 8)

The method as described in claim 8 wherein said D is input via a head tracking detector.

### Claim 13

An apparatus for generating a multi-screen rendering for a tilted multi-monitor display system, said apparatus comprising:
a processor and memory configured to execute instructions of an application that renders three-dimensional environments; and
a graphics processing unit communicatively coupled to the processor and the memory, wherein the graphics processing unit is configured to perform a method of generating a multi-screen rendering for the tilted multi-monitor display system, said method comprising:
defining a common viewport for a central monitor, a first peripheral monitor, and a second peripheral monitor, wherein the common viewport encapsulates final geometry to be displayed on the monitors, wherein the first peripheral monitor and the second peripheral monitor are tilted at an angle relative to the center monitor, and wherein the central monitor, the first peripheral monitor, and the second peripheral monitor are communicatively coupled to the graphics processing unit;
calculating scene data based on the common viewport and a viewing position to generate geometric primitives for the center monitor, the first peripheral monitor, and the second peripheral monitor;
rasterizing the geometric primitives for the center monitor, the first peripheral monitor, and the second peripheral monitor to generate respective bitmaps for the center monitor, the first peripheral monitor, and the second peripheral monitor;
adjusting the respective bitmaps at the pixel level for the first peripheral monitor and the second peripheral monitor based on the angle relative to the center monitor; and
rendering a perspective correct image on the center monitor, the first peripheral monitor, and the second peripheral monitor using the respective bitmaps.

### Claim 14 (depends on 13)

An apparatus as described in claim 13 wherein the adjusting the respective bitmaps for the first peripheral monitor and the second peripheral monitor based on the angle relative to the center monitor comprises performing at least one of oversampling and resampling pixels of the respective bitmaps.

### Claim 15 (depends on 13)

An apparatus as described in claim 13 further comprising obtaining a distance between the center monitor and a user and adjusting the respective bitmaps at the pixel level for the first peripheral monitor and the second peripheral monitor based on the distance.

### Claim 16 (depends on 13)

An apparatus as described in claim 13, further comprising determining a portion of the bitmaps that will not be visible based on the common viewport, wherein the rendering perspective correct images comprises skipping the portion of the bitmaps that will not be visible.

### Claim 17 (depends on 13)

An apparatus as described in claim 13, wherein said adjusting the respective bitmaps at the pixel level is performed by a plurality of triangle transformation engines, and wherein dedicated triangle transformation engines are associated with each of the center monitor, the first peripheral monitor, and the second peripheral monitor.

### Claim 18 (depends on 13)

An apparatus as described in claim 13 wherein said adjusting the respective bitmaps comprises performing image-space processing using a virtual plane that is tilted relative to the angle of the peripheral displays.

### Claim 19 (depends on 18)

An apparatus as described in claim 18 wherein said performing image-space processing comprises at least one of cropping and warping the bitmap.