# US 2025/0094825 A1 · Neural network architecture construction

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0094825 A1
- Document ID: US20250094825A1
- Status: Published application
- Kind: A1
- Application number: 18/468,603
- Filed: 2023-09-15
- Published: 2025-03-20
- Earliest claimed: 2023-09-15
- Assignees: NVIDIA Corporation
- Applicants: NVIDIA Corporation
- Inventors: Chong Yu
- Web page: https://patentgrep.com/patent/US20250094825A1

## CPC

- G06N 3/0985: Hyperparameter optimisation; Meta-learning; Learning-to-learn
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G05B 13/027: {using neural networks only}
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/045: Combinations of networks
- G06N 3/0495: Quantised networks; Sparse networks; Compressed networks
- G06N 3/063: using electronic means
- G06N 3/08: Learning methods
- G06N 3/082: modifying the architecture, e.g. adding, deleting or silencing nodes or connections
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)

## Abstract

Apparatuses, systems, and techniques to construct a neural network architecture. In at least one embodiment, candidate neural components may be selected for the neural network by jointly updating performance metric masks attached to these candidate neural components and a union neural network comprising all candidate neural components.

## Claims

### Claim 1

A processor comprising: one or more circuits to cause one or more portions of one or more neural networks to be selected to generate one or more neural networks based, at least in part, on one or more performance metric masks associated with the one or more portions.

### Claim 2 (depends on 1)

The processor of claim 1, wherein the one or more circuits further cause a construction of a union neural network comprising a union set of candidate neural components, and
wherein each of the one or more portions comprises one or more candidate neural components selected from the union set of candidate neural components.

### Claim 3 (depends on 2)

The processor of claim 2, wherein the one or more circuits further cause a training of the union neural network, and wherein the training comprises:
randomly masking a subset of weights of the union neural network,
computing a training loss based at least in part on a training output generated by the union neural network with randomly masked weights, and
updating unmasked weights of the union neural network based on the training loss during at least one training iteration

### Claim 4 (depends on 1)

The processor of claim 1, wherein at least one of the one or more performance metric masks is a multi-dimensional vector comprising multiple performance metrics of at least a portion of a neural network.

### Claim 5 (depends on 4)

The processor of claim 4, wherein at least a network accuracy metric from the multiple performance metrics is updated during training based at least in part on a training loss computed during a training iteration.

### Claim 6 (depends on 4)

The processor of claim 4, wherein the one or more performance metric masks associated with the one or more portions are updated during training by running the one or more portions with updated weights on a hardware simulator during a training iteration.

### Claim 7 (depends on 2)

The processor of claim 2, wherein the one or more portions of the one or more neural networks are selected based at least in part on the one or more performance metric masks that are updated during a training of the union neural network according to one or more target performance metrics.

### Claim 8

A system comprising: one or more processors to cause one or more portions of one or more neural networks to be selected to generate one or more neural networks based, at least in part, on one or more performance metric masks associated with the one or more portions.

### Claim 9 (depends on 8)

The system of claim 8, wherein the one or more processors further cause a construction of a union neural network comprising a union set of candidate neural components, and
wherein each of the one or more portions comprises one or more candidate neural components selected from the union set of candidate neural components.

### Claim 10 (depends on 9)

The system of claim 9, wherein the one or more processors further cause a training of the union neural network, and wherein the training comprises:
randomly masking a subset of weights of the union neural network,
computing a training loss based at least in part on a training output generated by the union neural network with randomly masked weights, and
updating unmasked weights of the union neural network based on the training loss during at least one training iteration

### Claim 11 (depends on 8)

The system of claim 8, wherein at least one of the one or more performance metric masks is a multi-dimensional vector comprising multiple performance metrics of at least a portion of a neural network.

### Claim 12 (depends on 11)

The system of claim 11, wherein at least a network accuracy metric from the multiple performance metrics is updated during training based at least in part on a training loss computed during a training iteration.

### Claim 13 (depends on 11)

The system of claim 11, wherein the one or more performance metric masks associated with the one or more portions are updated during training by running the one or more portions with updated weights on a hardware simulator during a training iteration.

### Claim 14 (depends on 9)

The system of claim 9, wherein the one or more portions of the one or more neural networks are selected based at least in part on the one or more performance metric masks that are updated during a training of the union neural network according to one or more target performance metrics.

### Claim 15

A machine-readable medium having stored thereon a set of instructions, which if performed by one or more processors, cause the one or more processors to at least: cause one or more portions of one or more neural networks to be selected to generate one or more neural networks based, at least in part, on one or more performance metric masks associated with the one or more portions.

### Claim 16 (depends on 15)

The medium of claim 15, wherein the one or more processors are further caused to cause a construction of a union neural network comprising a union set of candidate neural components, and
wherein each of the one or more portions comprises one or more candidate neural components selected from the union set of candidate neural components.

### Claim 17 (depends on 16)

The medium of claim 16, wherein the one or more processors are further caused to cause a training of the union neural network, and wherein the training comprises:
randomly masking a subset of weights of the union neural network,
computing a training loss based at least in part on a training output generated by the union neural network with randomly masked weights, and
updating unmasked weights of the union neural network based on the training loss during at least one training iteration

### Claim 18 (depends on 15)

The medium of claim 15, wherein at least one of the one or more performance metric masks is a multi-dimensional vector comprising multiple performance metrics of at least a portion of a neural network.

### Claim 19 (depends on 18)

The medium of claim 18, wherein at least a network accuracy metric from the multiple performance metrics is updated during training based at least in part on a training loss computed during a training iteration, and the one or more performance metric masks associated with the one or more portions are updated during training by running the one or more portions with updated weights on a hardware simulator during a training iteration.

### Claim 20 (depends on 19)

The medium of claim 19, wherein the one or more portions of the one or more neural networks are selected based at least in part on the one or more performance metric masks that are updated during a training of the union neural network according to one or more target performance metrics.
