# US 2023/0274395 A1 · System, devices and/or processes for application of machine learning to image anti-aliasing

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0274395 A1
- Document ID: US20230274395A1
- Status: Published application
- Kind: A1
- Application number: 17/681,443
- Filed: 2022-02-25
- Published: 2023-08-31
- Earliest claimed: 2022-02-25
- Assignees: Arm Limited
- Applicants: Arm Limited
- Inventors: Liam James O'Neil; Joshua James Sowerby; Matthew James Wash; Samuel James Edward Martin
- Web page: https://patentgrep.com/patent/US20230274395A1

## CPC

- G06N 3/09: Supervised learning
- G06T 5/70: Denoising; Smoothing
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06T 5/60: using machine learning, e.g. neural networks
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Example methods, apparatuses, and/or articles of manufacture are disclosed that may be implemented, in whole or in part, techniques to select between and/or among multiple available alternative approaches to perform a temporal anti-aliasing operation in processing an image.

## Claims

### Claim 1

A method comprising:
generating a densely sampled image frame and a sparse image frame based on a same rendered image frame, wherein the sparse image frame and the densely sampled image frame are at a resolution lower than a resolution of the same rendered image frame, and the sparse image frame is at a lower resolution than a resolution of the densely sampled image frame; and
applying the densely sampled image frame as a ground truth observation and applying the sparse image frame as a training parameter set in training a neural network to predict image signal intensity values.

### Claim 2 (depends on 1)

The method of claim 1, and further comprising:
mapping image signal intensity values of a plurality of pixel locations of the same rendered image frame to a plurality of sparse frames to generate the training parameter set.

### Claim 3 (depends on 2)

The method of claim 2, wherein mapping the image signal intensity values of the plurality of pixel locations to the plurality of sparse frames further comprises mapping image signal intensity of each pixel location in the same rendered image frame to exactly one of the sparse frames.

### Claim 4 (depends on 2)

The method of claim 2, wherein mapping the image signal intensity values of the plurality of pixel locations to the plurality of sparse frames further comprises selecting the image signal intensity values of the plurality of pixel locations so as to provide a low-discrepancy pattern.

### Claim 5 (depends on 1)

The method of claim 1, wherein generating the sparse image frame comprises decimating the same rendered image frame.

### Claim 6 (depends on 1)

The method of claim 1, and further comprising:
applying multiple instances of sparse frames based on the same rendered image frame as the training parameter set in a training iteration of the neural network to generate predicted accumulations of image signal values; and
updating weights associated with nodes of the neural network based, at least in part on application of a loss function to the densely sampled image frame and the predicted accumulations.

### Claim 7 (depends on 1)

The method of claim 1, wherein the neural network is trained to implement a super sampling to increase image resolution.

### Claim 8 (depends on 1)

The method of claim 1, wherein the neural network is trained to remove aliased edges and/or lines.

### Claim 9 (depends on 1)

The method of claim 1, wherein applying the densely sampled image frame as the ground truth observation and applying the sparse image frame as the training parameter set in training the neural network further comprises:
executing a sequence of iterations of the neural network, wherein one or more prediction values determined from one iteration of the neural network are provided as an input value to a subsequent iteration of the neural network.

### Claim 10 (depends on 1)

The method of claim 1, and further comprising:
sparsely sampling the same rendered image frame to provide a sparsely sampled image frame; and
jitter encoding the sparsely sampled image frame to provide the sparse image frame.

### Claim 11 (depends on 10)

The method of claim 10, wherein jitter encoding the sparsely sampled image frame comprises encoding one or more non-zero image signal intensity values in the sparsely sampled image frame according to corresponding pixel locations and magnitudes.

### Claim 12

An electronic device comprising:
a memory device; and
one or more processors coupled to the memory device to:
generate a densely sampled image frame and a sparse image frame based on a same rendered image frame, wherein the sparse image frame and the densely sampled image frame to be at a resolution lower than a resolution of the same rendered image frame, and the sparse image frame to be at a lower resolution than a resolution of the densely sampled image frame; and
apply the densely sampled image frame as a ground truth observation and apply the sparse image frame as a training parameter set in training a neural network to predict image signal intensity values.

### Claim 13 (depends on 12)

The electronic device of claim 12, wherein the one or more processors are further to:
map image signal intensity values of a plurality of pixel locations of the same rendered image frame to a plurality of sparse frames to generate the training parameter set.

### Claim 14 (depends on 13)

The electronic device of claim 13, wherein the one or more processors are further to:
map image signal intensity of each pixel location in the same rendered image frame to exactly one of the sparse frames.

### Claim 15 (depends on 13)

The electronic device of claim 13, wherein the one or more processors are further to map the image signal intensity values of the plurality of pixel locations to the plurality of sparse frames based, at least in part, on a selection of the image signal intensity values of the plurality of pixel locations so as to provide a low-discrepancy pattern.

### Claim 16 (depends on 12)

The electronic device of claim 12, wherein the one or more processors are further to:
apply multiple instances of sparse frames based on the same rendered image frame as the training parameter set in a training iteration of the neural network to generate predicted accumulations of image signal values; and
update weights associated with nodes of the neural network based, at least in part on application of a loss function to the densely sampled image frame and the predicted accumulations.

### Claim 17 (depends on 12)

The electronic device of claim 12, wherein the one or more processors are to apply the densely sampled image frame as the ground truth observation and apply the sparse image frame as the training parameter set in training the neural network based, at least in part, on execution of a sequence of iterations of the neural network, wherein one or more prediction values to be determined from one iteration of the neural network are to be provided as an input value to a subsequent iteration of the neural network.

### Claim 18 (depends on 12)

The electronic device of claim 12, and further comprising:
sparsely sampling the same rendered image frame to provide a sparsely sampled image frame; and
jitter encoding the sparsely sampled image frame to provide the sparse image frame.

### Claim 19 (depends on 18)

The electronic device of claim 18, wherein jitter encoding the sparsely sampled image frame comprises encoding one or more non-zero image signal intensity values in the sparsely sampled image frame according to corresponding pixel locations and magnitudes.

### Claim 20

An article comprising:
a storage medium comprising computer-readable instructions stored thereon, wherein the computer-readable instructions are executable by one or more processors of a computing device to:
generate a densely sampled image frame and a sparse image frame based on a same rendered image frame, wherein the sparse image frame and the densely sampled image frame to be at a resolution lower than a resolution of the same rendered image frame, and the sparse image frame to be at a lower resolution than a resolution of the densely sampled image frame; and
apply the densely sampled image frame as a ground truth observation and applying the sparse image frame as a training set in training a neural network to predict image signal intensity values.
