# US 2019/0318454 A1 · Adding greater realism to a computer-generated image by smoothing jagged edges

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0318454 A1
- Document ID: US20190318454A1
- Status: Published application
- Kind: A1
- Application number: 16/363,927
- Filed: 2019-03-25
- Published: 2019-10-17
- Earliest claimed: 2018-04-12
- Assignees: NVIDIA Corporation
- Applicants: NVIDIA Corporation
- Inventors: Adam Christopher Marrs; Josef B. Spjut; Holger Heinrich Gruen; Morgan McGuire; Rahul Sathe
- Web page: https://patentgrep.com/patent/US20190318454A1

## CPC

- G06T 5/70: Denoising; Smoothing
- G06T 15/005: {General purpose rendering architectures}
- G06T 5/50: using two or more images, e.g. averaging or subtraction
- G06T 7/50: Depth or shape recovery
- G06T 15/06: Ray-tracing
- G06T 15/503: {Blending, e.g. for anti-aliasing}
- G06T 2200/28: involving image processing hardware
- G06T 2207/20182: Noise reduction or smoothing in the temporal domain; Spatio-temporal filtering
- G06T 2207/20192: Edge enhancement; Edge preservation
- G06T 2207/30168: Image quality inspection
- G06T 2210/08: Bandwidth reduction
- G06T 2210/21: Collision detection, intersection
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

During the rendering of an image, specific pixels in the image are identified where antialiasing would be helpful. Antialiasing is then performed on these identified pixels, where anti-aliasing is a technique used to add greater realism to a digital image by smoothing jagged edges. This reduces a cost of performing antialiasing by reducing a number of pixels within an image on which antialiasing is performed.

## Claims

### Claim 1

A method comprising:
applying temporal antialiasing (TAA) to an image;
identifying failure pixels resulting from applying the TAA to the image; and
performing antialiasing on each of the failure pixels, utilizing a combination of ray tracing and fast approximate anti-aliasing (FXAA).

### Claim 2 (depends on 1)

The method of claim 1, wherein the failure pixels are identified utilizing results of applying the TAA to the image, utilizing intermediate data, or utilizing source geometry.

### Claim 3 (depends on 1)

The method of claim 1, wherein the image is created utilizing forward rendering or deferred rendering.

### Claim 4 (depends on 1)

The method of claim 1, wherein the failure pixels are identified during the TAA, where:
the TAA compares an historical value for a pixel in a previous frame to current values of neighboring pixels in a current frame, and
the pixel with the historical value that differs from the current values of neighboring pixels by more than a predetermined amount is identified as a failure pixel.

### Claim 5 (depends on 1)

The method of claim 1, wherein the failure pixels are identified by comparing a depth value for a pixel to depth values for neighboring pixels.

### Claim 6 (depends on 1)

The method of claim 1, wherein the failure pixels are identified by comparing a luminance value for a pixel to luminance values for neighboring pixels.

### Claim 7 (depends on 1)

The method of claim 1, wherein the failure pixels are marked using a sentinel value.

### Claim 8 (depends on 1)

The method of claim 1, wherein the failure pixels are included within a segmentation mask.

### Claim 9 (depends on 1)

The method of claim 1, wherein performing the antialiasing includes determining point-sampling visibility for a sub-pixel area within each of the failure pixels, utilizing GPU ray tracing.

### Claim 10 (depends on 1)

The method of claim 1, wherein results of performing the antialiasing on the failure pixels are combined with the results of applying the TAA to the image to create an output image.

### Claim 11 (depends on 1)

The method of claim 1, further including:
identifying one or more pixels where data does not exist in an original image, due to occlusion or lack of temporal data from a previous frame; and
performing antialiasing on the identified pixels, utilizing fast approximate anti-aliasing (FXAA).

### Claim 12

A system comprising:
a processor that is configured to:
apply temporal antialiasing (TAA) to an image;
identify failure pixels resulting from applying the TAA to the image; and
perform antialiasing on each of the failure pixels, utilizing a combination of ray tracing and fast approximate anti-aliasing (FXAA); and
a memory that is configured to store results of the antialiasing.

### Claim 13 (depends on 12)

The system of claim 12, wherein the failure pixels are identified utilizing results of applying the TAA to the image, utilizing intermediate data, or utilizing source geometry.

### Claim 14 (depends on 12)

The system of claim 12, wherein the image is created utilizing forward rendering or deferred rendering.

### Claim 15 (depends on 12)

The system of claim 12, wherein the failure pixels are identified during the TAA, where:
the TAA compares an historical value for a pixel in a previous frame to current values of neighboring pixels in a current frame, and
the pixel with the historical value that differs from the current values of neighboring pixels by more than a predetermined amount is identified as a failure pixel.

### Claim 16 (depends on 12)

The system of claim 12, wherein the failure pixels are identified by comparing a depth value for a pixel to depth values for neighboring pixels.

### Claim 17 (depends on 12)

The system of claim 12, wherein the failure pixels are identified by comparing a luminance value for a pixel to luminance values for neighboring pixels.

### Claim 18 (depends on 12)

The system of claim 12, wherein the failure pixels are marked using a sentinel value.

### Claim 19 (depends on 12)

The system of claim 12, wherein the failure pixels are included within a segmentation mask.

### Claim 20

A computer-readable storage medium storing instructions that, when executed by a processor, causes the processor to perform steps comprising:
applying temporal antialiasing (TAA) to an image;
identifying failure pixels resulting from applying the TAA to the image; and
performing antialiasing on each of the failure pixels, utilizing a combination of ray tracing and fast approximate anti-aliasing (FXAA).
