# US 2026/0099991 A1 · Converting neural radiance fields to 3d gaussians

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0099991 A1
- Document ID: US20260099991A1
- Status: Published application
- Kind: A1
- Application number: 19/351,781
- Filed: 2025-10-07
- Published: 2026-04-09
- Earliest claimed: 2024-10-08
- Assignees: NVIDIA Corporation
- Applicants: Nvidia Corporation
- Inventors: Kaiwen JIANG; Koki NAGANO; Shalini DE MELLO; Michael STENGEL
- Web page: https://patentgrep.com/patent/US20260099991A1

## CPC

- G06T 15/205: {Image-based rendering}
- G06T 17/00: Three-dimensional [3D] modelling for computer graphics
- G06T 15/06: Ray-tracing
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Converting neural radiance fields to three-dimensional (3D) Gaussians includes receiving multi-view images, projecting camera rays through each of the multi-view images, generating first 3D points by sampling each of the camera rays, generating first five-dimensional (5D) input coordinates from the first 3D points and a corresponding two-dimensional (2D) viewing direction, processing the first 5D input coordinates using a neural network to generate first 3D Gaussians, generating second 3D points by sampling each of the camera rays, generating second 5D input coordinates from the second 3D points and the corresponding 2D viewing direction, processing the second 5D input coordinates using the neural network to generate second 3D Gaussians, and pruning the first 3D Gaussians summed with the second 3D Gaussians.

## Claims

### Claim 1

A computer-implemented method for converting a neural radiance field to a plurality of three-dimensional (3D) Gaussians, the method comprising:
receiving a plurality of multi-view images;
projecting a plurality of camera rays through each of the plurality of multi-view images;
generating a plurality of first 3D points by sampling each of the plurality of camera rays;
generating a plurality of first five-dimensional (5D) input coordinates from the plurality of first 3D points and a corresponding two-dimensional (2D) viewing direction;
processing the plurality of first 5D input coordinates using a neural network to generate a plurality of first 3D Gaussians;
generating a plurality of second 3D points by sampling each of the plurality of camera rays;
generating a plurality of second 5D input coordinates from the plurality of second 3D points and the corresponding 2D viewing direction;
processing the plurality of second 5D input coordinates using the neural network to generate a plurality of second 3D Gaussians; and
pruning a sum of the plurality of first 3D Gaussians and the plurality of second 3D Gaussians to generate a plurality of pruned 3D Gaussians.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, wherein generating the plurality of first 3D points comprises uniformly sampling each of the plurality of camera rays.

### Claim 3 (depends on 1)

The computer-implemented method of claim 1, wherein generating the plurality of first 5D input coordinates comprises appending the corresponding 2D viewing direction of each 3D point of the plurality of first 3D points to the plurality of first 3D points.

### Claim 4 (depends on 1)

The computer-implemented method of claim 1, wherein generating the plurality of second 3D points comprises sampling each camera ray with samples biased towards regions of the plurality of multi-view images expected to contain visible content.

### Claim 5 (depends on 1)

The computer-implemented method of claim 1, wherein pruning the sum of the plurality of first 3D Gaussians and the plurality of second 3D Gaussians comprises pruning the sum of the plurality of first 3D Gaussians and the plurality of second 3D Gaussians to remove each of the plurality of the sum of the plurality of first 3D Gaussians and the plurality of second 3D Gaussians with color values below a given threshold.

### Claim 6 (depends on 1)

The computer-implemented method of claim 1, wherein the neural network comprises a multi-layer perceptron.

### Claim 7 (depends on 1)

The computer-implemented method of claim 1, further comprising generating a reconstructed 3D scene using the plurality of pruned 3D Gaussians.

### Claim 8 (depends on 7)

The computer-implemented method of claim 7, wherein generating the reconstructed 3D scene using the plurality of pruned 3D Gaussians comprises:
projecting each of the plurality of pruned 3D Gaussians to generate a plurality of rendered 2D images;
generating optimized 3D Gaussians from the plurality of rendered 2D images; and
generating the reconstructed 3D scene from the optimized 3D Gaussians.

### Claim 9 (depends on 8)

The computer-implemented method of claim 8, wherein projecting each of the plurality of pruned 3D Gaussians to generate the plurality of rendered 2D images comprises projecting each of the plurality of pruned 3D Gaussians onto a pixel-based image plane using a splatting-based rasterization technique.

### Claim 10 (depends on 8)

The computer-implemented method of claim 8, wherein generating the optimized 3D Gaussians comprises minimizing rendering loss between the plurality of rendered 2D images and corresponding images of the plurality of multi-view images.

### Claim 11 (depends on 8)

The computer-implemented method of claim 8, wherein generating the optimized 3D Gaussians further comprises fine tuning the optimized 3D Gaussians by removing optimized 3D Gaussians having an opacity value below a threshold.

### Claim 12 (depends on 8)

The computer-implemented method of claim 8, wherein generating the optimized 3D Gaussians further comprises fine tuning the optimized 3D Gaussians by densifying the optimized 3D Gaussians.

### Claim 13

One or more non-transitory computer-readable media storing instructions that, when executed by at least one processor, cause the at least one processor to perform the steps of:
receiving a plurality of multi-view images;
projecting a plurality of camera rays through each of the plurality of multi-view images;
generating a plurality of first 3D points by sampling each of the plurality of camera rays;
generating a plurality of first five-dimensional (5D) input coordinates from the plurality of first 3D points and a corresponding two-dimensional (2D) viewing direction;
processing the plurality of first 5D input coordinates using a neural network to generate a plurality of first 3D Gaussians;
generating a plurality of second 3D points by sampling each of the plurality of camera rays;
generating a plurality of second 5D input coordinates from the plurality of second 3D points and the corresponding 2D viewing direction;
processing the plurality of second 5D input coordinates using the neural network to generate a plurality of second 3D Gaussians; and
pruning a sum of the plurality of first 3D Gaussians and the plurality of second 3D Gaussians to generate a plurality of pruned 3D Gaussians.

### Claim 14 (depends on 13)

The one or more non-transitory computer-readable media of claim 13, wherein generating the plurality of first 3D points comprises uniformly sampling each of the plurality of camera rays.

### Claim 15 (depends on 13)

The one or more non-transitory computer-readable media of claim 13, wherein generating the plurality of second 3D points comprises sampling each camera ray with samples biased towards regions of the plurality of multi-view images expected to contain visible content.

### Claim 16 (depends on 13)

The one or more non-transitory computer-readable media of claim 13, wherein the steps further comprise generating a reconstructed 3D scene using the plurality of pruned 3D Gaussians.

### Claim 17 (depends on 13)

The one or more non-transitory computer-readable media of claim 13, wherein the neural network comprises a multi-layer perceptron.

### Claim 18 (depends on 13)

The one or more non-transitory computer-readable media of claim 13, wherein the steps further comprise generating a reconstructed 3D scene using the plurality of pruned 3D Gaussians by:
projecting each of the plurality of pruned 3D Gaussians to generate a plurality of rendered 2D images;
generating optimized 3D Gaussians from the plurality of rendered 2D images and generating the reconstructed 3D scene from the optimized 3D Gaussians.

### Claim 19 (depends on 18)

The one or more non-transitory computer-readable media of claim 18, wherein generating the optimized 3D Gaussians comprises minimizing rendering loss between the plurality of rendered 2D images and corresponding images of the plurality of multi-view images.

### Claim 20

A system, comprising:
one or more memories storing instructions; and
one or more processors that are coupled to the one or more memories and, when executing the instructions, are configured to perform steps comprising:
receiving a plurality of multi-view images;
projecting a plurality of camera rays through each of the plurality of multi-view images;
generating a plurality of first 3D points by sampling each of the plurality of camera rays;
generating a plurality of first five-dimensional (5D) input coordinates from the plurality of first 3D points and a corresponding two-dimensional (2D) viewing direction;
processing the plurality of first 5D input coordinates using a neural network to generate a plurality of first 3D Gaussians;
generating a plurality of second 3D points by sampling each of the plurality of camera rays;
generating a plurality of second 5D input coordinates from the plurality of second 3D points and the corresponding 2D viewing direction;
processing the plurality of second 5D input coordinates using the neural network to generate a plurality of second 3D Gaussians; and
pruning a sum of the plurality of first 3D Gaussians and the plurality of second 3D Gaussians to generate a plurality of pruned 3D Gaussians.
