# US 2022/0036157 A1 · Gateway for distributing an artificial neural network among multiple processing nodes

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0036157 A1
- Document ID: US20220036157A1
- Status: Published application
- Kind: A1
- Application number: 16/942,341
- Filed: 2020-07-29
- Published: 2022-02-03
- Earliest claimed: 2020-07-29
- Assignees: Micron Technology, Inc.
- Applicants: Micron Technology, Inc.
- Inventors: Poorna Kale; Amit Gattani
- Web page: https://patentgrep.com/patent/US20220036157A1

## CPC

- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/0495: Quantised networks; Sparse networks; Compressed networks
- G06N 3/063: using electronic means
- G06N 3/082: modifying the architecture, e.g. adding, deleting or silencing nodes or connections
- G06N 3/09: Supervised learning
- G06N 3/092: Reinforcement learning
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Methods, systems, and apparatus related to dynamic distribution of an artificial neural network among multiple processing nodes based on real-time monitoring of a processing load on each node. In one approach, a server acts as an intelligent artificial intelligence (AI) gateway. The server receives data regarding a respective operating status for each of monitored processing devices. The monitored processing devices perform processing for an artificial neural network (ANN). The monitored processing devices each perform processing for a portion of the neurons in the ANN. The portions are distributed in response to monitoring the processing load on each processing device (e.g., to better utilize processing power across all of the processing devices).

## Claims

### Claim 1

A system comprising:
at least one processing device; and
at least one memory containing instructions configured to instruct the at least one processing device to:
receive data regarding a respective operating status for each of monitored processing devices, wherein the monitored processing devices perform processing for neurons in an artificial neural network (ANN), and wherein the monitored processing devices include a first processing device that performs processing for first neurons of the ANN;
determine, based on the received data, a new distribution for the neurons in the ANN, wherein the new distribution indicates that a second processing device is to perform processing for the first neurons; and
in response to determining the new distribution, send a communication to the second processing device to transfer processing for the first neurons from the first processing device to the second processing device.

### Claim 2 (depends on 1)

The system of claim 1, wherein the ANN is used to process data received from at least one sensing device.

### Claim 3 (depends on 2)

The system of claim 2, wherein the new distribution corresponds to a change in an operating context of the at least one sensing device.

### Claim 4 (depends on 2)

The system of claim 2, wherein a vehicle includes the at least one sensing device and the second processing device, and wherein the second processing device controls at least one function of the vehicle.

### Claim 5 (depends on 1)

The system of claim 1, wherein:
the ANN is a first ANN;
the received data is input to a second ANN;
processing for the second ANN is performed by the at least one processing device; and
the determining the new distribution comprises predicting the new distribution based on output from the second ANN.

### Claim 6 (depends on 1)

The system of claim 1, wherein the sent communication comprises at least one of neuron connections or neuron weights for the first neurons.

### Claim 7 (depends on 1)

The system of claim 1, wherein the instructions are further configured to instruct the at least one processing device to send a communication to the first processing device regarding the transfer of processing for the first neurons from the first processing device to the second processing device.

### Claim 8 (depends on 1)

The system of claim 1, wherein the instructions are further configured to instruct the at least one processing device to:
add a new processing device to perform processing for the ANN; and
in response to determining the new distribution, send a communication to the new processing device to transfer processing for second neurons of the ANN to the new processing device.

### Claim 9 (depends on 1)

The system of claim 1, wherein at least one data source provides data used as input to the ANN, and wherein the new distribution is determined based on a proximity of the monitored processing devices to the at least one data source.

### Claim 10 (depends on 1)

The system of claim 1, wherein the at least one processing device comprises at least one of a neural network accelerator, a field programmable gate array (FPGA), a central processing unit (CPU), or a graphics processing unit (GPU).

### Claim 11

A method comprising:
monitoring, by a first processing device, processing devices performing processing for neurons in an artificial neural network (ANN), wherein the monitored processing devices include a second processing device that performs processing for first neurons of the ANN;
receiving data regarding a respective operating status for each of the monitored processing devices;
determining, based on the received data, a new distribution for the neurons in the ANN, wherein the new distribution indicates that a third processing device is to perform processing for the first neurons; and
transferring, based on the new distribution, processing for the first neurons from the second processing device to the third processing device.

### Claim 12 (depends on 11)

The method of claim 11, wherein the new distribution comprises at least one of neuron connections or neuron weights for the first neurons.

### Claim 13 (depends on 12)

The method of claim 12, wherein the transferring processing comprises:
updating a configuration file to include the at least one of neuron connections or neuron weights; and
sending the updated configuration file to the first processing device and the second processing device.

### Claim 14 (depends on 11)

The method of claim 11, wherein each respective operating status comprises at least one of available resources of the monitored processing device, an extent of resource utilization by the monitored processing device, a data flow to the monitored processing device, or a data flow from the monitored processing device.

### Claim 15 (depends on 11)

The method of claim 11, wherein:
the ANN is a first ANN;
the received data is input to a second ANN; and
the determining the new distribution for the neurons in the first ANN is based on output from the second ANN.

### Claim 16 (depends on 11)

The method of claim 11, wherein output from the ANN is used to control a vehicle, and the new distribution corresponds to a change in an operating context of the vehicle.

### Claim 17 (depends on 16)

The method of claim 16, wherein the change in the operating context is at least one of a change in lighting conditions in which the vehicle is operating, a change in weather conditions in which the vehicle is operating, a change in a configuration of functions performed by the vehicle, or a change in a number of processing devices available to perform processing for the neurons in the ANN.

### Claim 18

At least one non-transitory computer-readable medium storing instructions which, when executed on at least one computing device, cause the at least one computing device to at least:
monitor processing devices performing processing for neurons in an artificial neural network (ANN), wherein the monitored processing devices include a first processing device that performs processing for first neurons of the ANN;
determine, based on monitoring the processing devices, a new distribution for the ANN; and
transfer, based on the new distribution, processing for the first neurons from the first processing device to a second processing device.

### Claim 19 (depends on 18)

The non-transitory computer-readable medium of claim 18, wherein the new distribution is determined based on at least one predicted data flow between the monitored processing devices associated with at least one of configuring the ANN based on the new distribution, or performing processing for the neurons in the ANN based on the new distribution.

### Claim 20 (depends on 18)

The non-transitory computer-readable medium of claim 18, wherein the determining the new distribution is based on predicted activity of at least one neuron in the ANN.
