# US 12,602,751 B2 · Sample distribution-informed denoising and rendering

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,602,751 B2
- Document ID: US12602751B2
- Status: Granted
- Kind: B2
- Application number: 17/520,089
- Filed: 2021-11-05
- Published: 2023-03-02
- Granted: 2026-04-14
- Earliest claimed: 2021-08-19
- Assignees: Intel Corporation
- Applicants: Intel Corporation
- Inventors: Tobias Zirr; Sungye Kim
- Web page: https://patentgrep.com/patent/US12602751B2

## CPC

- G06T 5/70: Denoising; Smoothing
- G06N 3/04: Architecture, e.g. interconnection topology
- G06T 1/20: Processor architectures; Processor configuration, e.g. pipelining
- G06T 3/4053: based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
- G06T 5/50: using two or more images, e.g. averaging or subtraction
- G06T 5/60: using machine learning, e.g. neural networks
- G06V 10/82: using neural networks
- G06V 20/20: in augmented reality scenes
- G06T 2200/12: involving antialiasing
- G06T 2207/10016: Video; Image sequence
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A graphics processor is provided that includes circuitry configured to receive, at an input block of a neural network model, a set of data including previous frame data, current frame data, velocity data, and jitter offset data. The neural network model is configured to generate a denoised, supersampled, and anti-aliased output image based on reliability metrics computed based on sample distribution data for samples within the current frame data.

## Claims

### Claim 1

A graphics processor comprising:
a set of processing resources configured to perform a supersampling anti-aliasing operation via a mixed precision convolutional neural network, the set of processing resources including circuitry configured to:
receive, at an input block of a neural network model, a set of data including previous frame data, current frame data, velocity data, and jitter offset data;
pre-process the set of data to generate pre-processed data;
provide first pre-processed data to a feature extraction network of the neural network model and second pre-processed data to an output block of the neural network model, the first pre-processed data in a datatype having a first precision and the second pre-processed data in a datatype having a second precision that is higher than the first precision;
process the pre-processed data at the feature extraction network via one or more encoder stages and one or more decoder stages;
output tensor data from the feature extraction network to the output block; and
generate an output image via an output block of the neural network model, wherein the output image is a denoised, supersampled, and anti-aliased output image and the output block is configured to filter the output image based on sample reliability metrics computed based on sample distribution data for samples within the current frame data.

### Claim 2 (depends on 1)

The graphics processor as in claim 1, wherein the circuitry is configured to receive, at the input block, the sample reliability metrics and to filter the output image, the circuitry is configured to denoise the output image based on the sample reliability metrics.

### Claim 3 (depends on 2)

The graphics processor as in claim 2, wherein the current frame data includes high dynamic range (HDR) color data.

### Claim 4 (depends on 3)

The graphics processor as in claim 3, wherein the current frame data includes G-buffer data including auxiliary image data.

### Claim 5 (depends on 4)

The graphics processor as in claim 4, wherein the auxiliary image data includes albedo and normal data and the sample reliability metrics are configured at least in part to compensate for noise within the auxiliary image data.

### Claim 6 (depends on 5)

The graphics processor as in claim 5, wherein the sample reliability metrics include a moment of a statistical distribution of sample characteristics for samples within the current frame data.

### Claim 7 (depends on 6)

The graphics processor as in claim 6, wherein the sample reliability metrics are computed based on a perceptually uniform encoding of the samples in which distortion visibility is approximately uniform along encoded values.

### Claim 8 (depends on 7)

The graphics processor as in claim 7, wherein variance of the sample distribution data is characterized by a Jensen gap of perceptually uniform, tone-mapped samples.

### Claim 9 (depends on 8)

The graphics processor as in claim 8, wherein the circuitry includes a matrix accelerator that is configured to perform matrix operations for the neural network model.

### Claim 10 (depends on 9)

The graphics processor as in claim 9, wherein the matrix accelerator includes a systolic array.

### Claim 11

A method comprising:
on a graphics processor device:
receiving, at an input block of a neural network model, a set of data including previous frame data, current frame data, velocity data, and jitter offset data;
pre-processing the set of data to generate pre-processed data;
providing first pre-processed data to a feature extraction network of the neural network model and second pre-processed data to an output block of the neural network model, the first pre-processed data in a datatype having a first precision and the second pre-processed data in a datatype having a second precision that is higher than the first precision;
processing the pre-processed data at the feature extraction network via one or more encoder stages and one or more decoder stages;
outputting tensor data from the feature extraction network to the output block; and
generating an output image via an output block of the neural network model, wherein the output image is a denoised, supersampled, and anti-aliased output image and the output block is configured to filter the output image based on sample reliability metrics computed based on sample distribution data for samples within the current frame data.

### Claim 12 (depends on 11)

The method as in claim 11, further comprising receiving, at the input block, the sample reliability metrics for the current frame data, wherein filtering the output image includes denoising the output image based on the sample reliability metrics, wherein, and the sample reliability metrics include a moment of a statistical distribution of sample characteristics for samples within the current frame data.

### Claim 13 (depends on 12)

The method as in claim 12, wherein the current frame data includes high dynamic range (HDR) color data.

### Claim 14 (depends on 13)

The method as in claim 13, wherein the current frame data includes G-buffer data including auxiliary image data.

### Claim 15 (depends on 14)

The method as in claim 14, wherein the auxiliary image data includes albedo and normal data and the sample reliability metrics are configured at least in part to compensate for noise within the auxiliary image data.

### Claim 16

A non-transitory machine readable medium storing instructions which, when executed by one or more processors including a graphics processor, cause the one or more processors to perform operations comprising:
receiving, at an input block of a neural network model, a set of data including previous frame data, current frame data, velocity data, jitter offset data, and sample reliability metrics determined based on sample distribution data for samples within the current frame data;
pre-processing the set of data to generate pre-processed data;
providing first pre-processed data to a feature extraction network of the neural network model and second pre-processed data to an output block of the neural network model, the first pre-processed data in a datatype having a first precision and the second pre-processed data in a datatype having a second precision that is higher than the first precision;
processing the pre-processed data at the feature extraction network via one or more encoder stages and one or more decoder stages;
outputting tensor data from the feature extraction network to the output block; and
generating an output image via an output block of the neural network model, wherein the output image is a denoised, supersampled, and anti-aliased output image and the output block is configured to filter the output image based on the sample reliability metrics.

### Claim 17 (depends on 16)

The non-transitory machine readable medium as in claim 16, wherein filtering the output image includes denoising the output image based on sample reliability metrics, wherein the sample reliability metrics include a moment of a statistical distribution of sample characteristics for samples within the current frame data.

### Claim 18 (depends on 17)

The non-transitory machine readable medium as in claim 17, wherein the current frame data includes high dynamic range (HDR) color data.

### Claim 19 (depends on 18)

The non-transitory machine readable medium as in claim 18, wherein the current frame data includes G-buffer data including auxiliary image data.

### Claim 20 (depends on 19)

The non-transitory machine readable medium as in claim 19, wherein the auxiliary image data includes albedo and normal data and the sample reliability metrics are configured at least in part to compensate for noise within the auxiliary image data.