# US 2025/0156715 A1 · Neural Architecture Search with Improved Computational Efficiency

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0156715 A1
- Document ID: US20250156715A1
- Status: Published application
- Kind: A1
- Application number: 18/839,379
- Filed: 2022-12-27
- Published: 2025-05-15
- Earliest claimed: 2022-02-16
- Assignees: Google Inc.
- Applicants: Google LLC
- Inventors: Da Huang; Chengrun Yang; Pieter-Jan Kindermans; Hanxiao Liu; Quoc V. Le; Madeleine Richards Udell; Yifeng Lu; Gabriel Mintzer Bender
- Web page: https://patentgrep.com/patent/US20250156715A1

## CPC

- G06N 3/082: modifying the architecture, e.g. adding, deleting or silencing nodes or connections
- G06N 3/0499: Feedforward networks
- G06N 3/092: Reinforcement learning
- G06N 3/0985: Hyperparameter optimisation; Meta-learning; Learning-to-learn
- G06N 3/048: Activation functions
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/096: Transfer learning
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Provided are neural architecture search techniques that have improved computational efficiency via performance of an initial constraint evaluation and improved gradient update approach. Further, the proposed approaches provide significant improvements for certain modalities of input data, such as tabular datasets.

## Claims

### Claim 1

A computer-implemented method of neural architecture search with increased computational efficiency, the method comprising:
defining, by a computing system comprising one or more computing devices, a plurality of searchable parameters that control an architecture of a neural network, wherein the neural network is configured to process input data to produce inferences; and
for one or more iterations:
determining, by the computing system using a controller model, a new set of values for the plurality of searchable parameters to generate a new architecture for the neural network;
determining, by the computing system, whether the neural network with the new architecture satisfies one or more constraints;
when the neural network with the new architecture does not satisfy the one or more constraints: discarding, by the computing system, the new architecture; and
when the neural network with the new architecture satisfies the one or more constraints:
determining, by the computing system, one or more performance metrics for the neural network with the new architecture relative to production of inferences for a set of validation data;
evaluating, by the computing system, a value function that provides a value based at least in part on the one or more performance metrics and a conditional probability of the new architecture for the neural network given that the new architecture for the neural network satisfies the one or more constraints; and
updating, by the computing system, one or more values of one or more parameters of the controller model based on the value function.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, wherein the conditional probability comprises an exact conditional probability.

### Claim 3 (depends on 1)

The computer-implemented method of claim 1, wherein the conditional probability comprises an estimated conditional probability.

### Claim 4 (depends on 3)

The computer-implemented method of claim 3, wherein evaluating, by the computing system, the value function comprises performing, by the computing system, a Monte-Carlo sampling technique to determine the estimated conditional probability.

### Claim 5 (depends on 1)

The computer-implemented method of claim 1, wherein:
the one or more constraints comprise a size constraint that requires that a number of parameters included in the new network architecture does not exceed a threshold number of parameters.

### Claim 6 (depends on 1)

The computer-implemented method of claim 1, wherein:
the one or more constraints comprise a training latency constraint that requires that training of neural network with the new architecture does not exceed a threshold training time.

### Claim 7 (depends on 1)

The computer-implemented method of claim 1, wherein:
the one or more constraints comprise a runtime latency constraint that requires that ta runtime latency of neural network with the new architecture does not exceed a threshold runtime.

### Claim 8 (depends on 1)

The computer-implemented method of claim 1, wherein the validation data comprises tabular data.

### Claim 9 (depends on 1)

The computer-implemented method of claim 1, wherein discarding, by the computing system, the new architecture comprises discarding, by the computing system, the new architecture prior to completion of training of a neural network having the new architecture.

### Claim 10 (depends on 1)

The computer-implemented method of claim 1, wherein the controller model comprises a plurality of sets of logits that are respectively associated with the plurality of searchable parameters, and wherein each of the plurality of sets of logits generates its respective prediction independent of the other sets of logits.

### Claim 11

A computer system, comprising:
one or more processors; and
one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors, cause the computing system to perform operations, the operations comprising:
determining, by the computing system using a controller model, a new set of values for a plurality of searchable parameters to generate a new architecture for the neural network;
determining, by the computing system, one or more performance metrics for the neural network with the new architecture relative to production of inferences for a set of validation data;
evaluating, by the computing system, a value function that provides a value based at least in part on the one or more performance metrics and a conditional probability of the new architecture for the neural network given that the new architecture for the neural network satisfies one or more constraints; and
updating, by the computing system, one or more values of one or more parameters of the controller model based on the value function.

### Claim 12 (depends on 11)

The computer system of claim 11, wherein the conditional probability comprises an exact conditional probability.

### Claim 13 (depends on 11)

The computer system of claim 11, wherein the conditional probability comprises an estimated conditional probability.

### Claim 14 (depends on 13)

The computer system of claim 13, wherein evaluating, by the computing system, the value function comprises performing, by the computing system, a Monte-Carlo sampling technique to determine the estimated conditional probability.

### Claim 15 (depends on 11)

The computer system of any of claim 11, wherein:
the one or more constraints comprise a size constraint that requires that a number of parameters included in the new network architecture does not exceed a threshold number of parameters.

### Claim 16 (depends on 11)

The computer system of any of claim 11, wherein:
the one or more constraints comprise a training latency constraint that requires that training of neural network with the new architecture does not exceed a threshold training time.

### Claim 17 (depends on 11)

The computer system of any of claim 11, wherein:
the one or more constraints comprise a runtime latency constraint that requires that ta runtime latency of neural network with the new architecture does not exceed a threshold runtime.

### Claim 18 (depends on 11)

The computer system of any of claim 11, wherein the validation data comprise tabular data.

### Claim 19

One or more non-transitory computer-readable media that collectively store a neural network having a final architecture identified by performance of operations for a plurality of iterations, the operations comprising:
determining, by the computing system using a controller model, a new set of values for a plurality of searchable parameters to generate a new architecture for the neural network;
determining, by the computing system, one or more performance metrics for the neural network with the new architecture relative to production of inferences for a set of validation data;
evaluating, by the computing system, a value function that provides a value based at least in part on the one or more performance metrics and a conditional probability of the new architecture for the neural network given that the new architecture for the neural network satisfies one or more constraints; and
updating, by the computing system, one or more values of one or more parameters of the controller model based on the value function.

### Claim 20 (depends on 19)

The one or more non-transitory computer-readable media of claim 19, wherein the training data and the validation data comprise tabular data.
