# US 2023/0274130 A1 · Hardware-assisted gradient optimization using streamed gradients

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0274130 A1
- Document ID: US20230274130A1
- Status: Published application
- Kind: A1
- Application number: 18/311,670
- Filed: 2023-05-03
- Published: 2023-08-31
- Earliest claimed: 2020-03-23
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Jinwen XI; Bharadwaj PUDIPEDDI; Marc TREMBLAY
- Web page: https://patentgrep.com/patent/US20230274130A1

## CPC

- G06N 3/063: using electronic means
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/09: Supervised learning
- G06N 3/098: Distributed learning, e.g. federated learning
- G06N 5/046: Forward inferencing; Production systems
- G06N 20/00: Machine learning
- G11C 11/34: using semiconductor devices
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G11C: STATIC STORES (semiconductor memory devices H10B)

## Abstract

Systems and methods related to hardware-assisted gradient optimization using streamed gradients are described. An example method in a system comprising a memory configured to store weights associated with a neural network model comprising L layers, where L is an integer greater than one, a gradient optimizer, and a plurality of workers is described. The method includes during a single burst cycle moving a first set of gradients, received from each of the plurality of workers, from at least one gradient buffer to the gradient optimizer and moving weights from at least one buffer, coupled to the memory, to the gradient optimizer. The method further includes during the single burst cycle writing back the new weights, calculated by the gradient optimizer, to the memory. The method further includes during the single burst cycle transmitting the new weights, from the gradient optimizer, to each of the plurality of workers.

## Claims

### Claim at position 1

1-20. (canceled)

### Claim 21

A method comprising:
during a single burst cycle, receiving gradients from each of a plurality of workers, into a predetermined number of gradient buffers;
during the single burst cycle, providing the received gradients to a reduction block to generate reduced gradients;
during the single burst cycle, providing the reduced gradients to a gradient optimizer data path associated with the gradient optimizer;
during the single burst cycle, moving weights from at least one buffer, coupled to a memory, to the gradient optimizer;
during the single burst cycle writing back new weights, calculated by the gradient optimizer, to the memory; and
during the single burst cycle transmitting the new weights from the gradient optimizer to each of the plurality of workers, wherein during the single burst cycle the gradient optimizer operates on a gradient burst having a fixed number of gradients.

### Claim 22 (depends on 21)

The method of claim 21, wherein the successive burst cycles result in streaming of gradients through the gradient optimizer.

### Claim 23 (depends on 21)

The method of claim 21, wherein the fixed number of gradients is dependent at least upon a selected common depth of the predetermined number of gradient buffers.

### Claim 24 (depends on 21)

The method of claim 21, wherein the at least one buffer, coupled to the memory, has a selected depth equal to or greater than the selected common depth of the predetermined number of gradient buffers.

### Claim 25 (depends on 21)

The method of claim 21, wherein during each burst cycle, the gradient optimizer is configured to operate only on gradients buffered in the predetermined number of gradient buffers.

### Claim 26 (depends on 25)

The method of claim 25, wherein the gradients buffered in the predetermined number of gradient buffers are discarded after processing by the gradient optimizer.

### Claim 27 (depends on 21)

The method of claim 21, wherein the gradient optimizer is configured to implement any one of a plurality of optimization algorithms, wherein the plurality of optimization algorithms comprises stochastic gradient descent (SGD) algorithm, SGD with momentum algorithm, and adaptive moment estimation algorithm.

### Claim 28

A system comprising:
a memory coupled to at least one buffer;
a gradient optimizer coupled to the memory via the at least one buffer,
wherein the system is configured to:
during a single burst cycle, receiving gradients from each of a plurality of workers, into a predetermined number of gradient buffers associated with the gradient optimizer,
during the single burst cycle, provide the received gradients to a reduction block to generate reduced gradients,
during the single burst cycle, provide the reduced gradients to a gradient optimizer data path associated with the gradient optimizer,
during the single burst cycle, move weights from at least one buffer, coupled to the memory, to the gradient optimizer,
during the single burst cycle write back new weights, calculated by the gradient optimizer, to the memory, and
during the single burst cycle transmit the new weights from the gradient optimizer to each of the plurality of workers, wherein during the single burst cycle the gradient optimizer operates on a gradient burst having a fixed number of gradients.

### Claim 29 (depends on 28)

The system of claim 28, wherein the successive burst cycles result in streaming of gradients through the gradient optimizer.

### Claim 30 (depends on 28)

The system of claim 28, wherein the fixed number of gradients is dependent at least upon a selected common depth of the predetermined number of gradient buffers.

### Claim 31 (depends on 28)

The system of claim 28, wherein the at least one buffer, coupled to the memory, has a selected depth equal to or greater than the selected common depth of the predetermined number of gradient buffers.

### Claim 32 (depends on 28)

The system of claim 28, wherein during each burst cycle, the gradient optimizer is configured to operate only on gradients buffered in the predetermined number of gradient buffers.

### Claim 33 (depends on 32)

The system of claim 32, wherein the gradients buffered in the predetermined number of gradient buffers are discarded after processing by the gradient optimizer.

### Claim 34 (depends on 28)

The system of claim 28, wherein the gradient optimizer is configured to implement any one of a plurality of optimization algorithms; wherein the plurality of optimization algorithms comprises stochastic gradient descent (SGD) algorithm, SOD with momentum algorithm, and adaptive moment estimation algorithm.

### Claim 35

A method for operating a system comprising a memory configured to store weights associated with a neural network model comprising L layers, wherein L is an integer greater than one, a gradient optimizer, and a plurality of workers, wherein each of the plurality of workers is configured to perform a forward pass and a backward pass on any one of the L layers associated with the neural network model, the method comprising:
during a single burst cycle, receiving gradients from each of the plurality of workers, into a predetermined number of gradient buffers;
during the single burst cycle, providing the received gradients to a reduction block to generate reduced gradients;
during the single burst cycle, providing the reduced gradients to a gradient optimizer data path associated with the gradient optimizer;
during the single burst cycle, moving weights from at least one buffer, coupled to the memory, to the gradient optimizer;
during the single burst cycle writing back new weights, calculated by the gradient optimizer, to the memory; and
during the single burst cycle transmitting the new weights from the gradient optimizer to each of the plurality of workers, wherein during the single burst cycle the gradient optimizer operates on a gradient burst having a fixed number of gradients.

### Claim 36 (depends on 35)

The method of claim 35, wherein the successive burst cycles result in streaming of gradients through the gradient optimizer.

### Claim 37 (depends on 36)

The method of claim 36, wherein the fixed number of gradients is dependent at least upon a selected common depth of the predetermined number of gradient buffers, and wherein the at least one buffer, coupled to the memory, has a selected depth equal to or greater than the selected common depth of the predetermined number of gradient buffers.

### Claim 38 (depends on 36)

The method of claim 36, wherein during each burst cycle, the gradient optimizer is configured to operate only on gradients buffered in the predetermined number of gradient buffers.

### Claim 39 (depends on 38)

The method of claim 38, wherein the gradients buffered in the predetermined number of gradient buffers are discarded after processing by the gradient optimizer.

### Claim 40 (depends on 35)

The method of claim 35, wherein the gradient optimizer is configured to implement any one of a plurality of optimization algorithms, wherein the plurality of optimization algorithms comprises stochastic gradient descent (SGD) algorithm, SGD with momentum algorithm, and adaptive moment estimation algorithm.
