# US 2023/0146611 A1 · Neural network training with acceleration

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0146611 A1
- Document ID: US20230146611A1
- Status: Published application
- Kind: A1
- Application number: 17/668,345
- Filed: 2022-02-09
- Published: 2023-05-11
- Earliest claimed: 2021-11-11
- Assignees: Samsung Electronics Co., Ltd.
- Applicants: Samsung Electronics Co., Ltd.
- Inventors: Shiyu LI; Krishna T. MALLADI; Andrew CHANG; Yang Seok KI
- Web page: https://patentgrep.com/patent/US20230146611A1

## CPC

- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06T 1/20: Processor architectures; Processor configuration, e.g. pipelining
- G06F 9/30036: {Instructions to perform operations on packed data, e.g. vector, tile or matrix operations}
- G06F 13/4022: {using switching circuits, e.g. switching matrix, connection or expansion network (G06F13/4009 takes precedence)}
- G06F 18/2148: {characterised by the process organisation or structure, e.g. boosting cascade}
- G06N 3/045: Combinations of networks
- G06N 3/0499: Feedforward networks
- G06N 3/063: using electronic means
- G06N 3/09: Supervised learning
- G06T 1/60: Memory management
- G06F 12/0815: Cache consistency protocols
- G06F 2212/1016: Performance improvement
- G06F 2212/454: Vector or matrix data
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A system and method for training a neural network. In some embodiments, the system includes a computational storage device including a backing store. The computational storage device may be configured to: store, in the backing store, an embedding table for a neural network embedding operation; receive a first index vector including a first index and a second index; retrieve, from the backing store: a first row of the embedding table, corresponding to the first index, and a second row of the embedding table, corresponding to the second index; and calculate a first embedded vector based on the first row and the second row.

## Claims

### Claim 1

A system, comprising:
a computational storage device comprising a backing store,
wherein the computational storage device is configured to:
store, in the backing store, an embedding table for a neural network embedding operation;
receive a first index vector comprising a first index and a second index;
retrieve, from the backing store:
a first row of the embedding table, corresponding to the first index, and
a second row of the embedding table, corresponding to the second index; and
calculate a first embedded vector based on the first row and the second row.

### Claim 2 (depends on 1)

The system of claim 1, further comprising a graphics processing unit, wherein:
the computational storage device further comprises a memory; and
the computational storage device is further configured to expose a portion of the memory for access by the graphics processing unit.

### Claim 3 (depends on 2)

The system of claim 2, wherein:
the computational storage device is further configured to store the first embedded vector in the portion of the memory; and
the graphics processing unit is configured to read the first embedded vector from the portion of the memory.

### Claim 4 (depends on 1)

The system of claim 1, wherein the computational storage device is further configured to:
receive a gradient vector; and
update a row of the embedding table based on the gradient vector to form an updated row.

### Claim 5 (depends on 4)

The system of claim 4, wherein the computational storage device is further configured to update the first embedded vector, based on the gradient vector, to form an updated first embedded vector.

### Claim 6 (depends on 5)

The system of claim 5, wherein the updating of the first embedded vector comprises:
calculating an overlap count, the overlap count being a count of elements present both in the first index vector and in a previous batch of index vectors.

### Claim 7 (depends on 6)

The system of claim 6, wherein the calculating of the first embedded vector comprises summing the first row and the second row.

### Claim 8 (depends on 7)

The system of claim 7, wherein the updating of the first embedded vector further comprises adding to the first embedded vector the product of:
the overlap count,
the gradient vector, and
a learning rate.

### Claim 9 (depends on 1)

The system of claim 1, wherein:
a second index vector comprises the first index; and
the computational storage device is further configured to:
store the first row in a cache of the computational storage device, and
calculate a second embedded vector based on the first row.

### Claim 10

A method, comprising:
storing, by a computational storage device, in a backing store of the computational storage device, an embedding table for a neural network embedding operation;
receiving, by the computational storage device, a first index vector comprising a first index and a second index;
retrieving, by the computational storage device, from the backing store, to a first cache of the computational storage device:
a first row of the embedding table, corresponding to the first index, and
a second row of the embedding table, corresponding to the second index; and
calculating, by the computational storage device, a first embedded vector based on the first row and the second row.

### Claim 11 (depends on 10)

The method of claim 10, further comprising:
receiving a gradient vector; and
updating a row of the embedding table based on the gradient vector to form an updated row.

### Claim 12 (depends on 11)

The method of claim 11, further comprising updating the first embedded vector, based on the gradient vector, to form an updated first embedded vector.

### Claim 13 (depends on 12)

The method of claim 12, wherein the updating of the first embedded vector comprises:
calculating an overlap count, the overlap count being a count of elements present both in the first index vector and in a previous batch of index vectors.

### Claim 14 (depends on 13)

The method of claim 13, wherein the calculating of the first embedded vector comprises summing the first row and the second row.

### Claim 15 (depends on 14)

The method of claim 14, wherein the updating of the first embedded vector further comprises adding to the first embedded vector the product of:
the overlap count,
the gradient vector, and
a learning rate.

### Claim 16 (depends on 15)

The method of claim 15, further comprising storing the updated first embedded vector in a second cache.

### Claim 17 (depends on 13)

The method of claim 13, wherein:
a second index vector comprises the first index; and
the method further comprises:
storing the first row in the first cache, and
calculating a second embedded vector based on the first row.

### Claim 18 (depends on 11)

The method of claim 11, further comprising storing the updated row in the backing store.

### Claim 19

A system, comprising:
a computational storage device comprising:
means for processing;
a first cache; and
a backing store,
wherein the means for processing is configured to:
store, in the backing store, an embedding table for a neural network embedding operation;
receive a first index vector comprising a first index and a second index;
retrieve, from the backing store, to the first cache:
a first row of the embedding table, corresponding to the first index, and
a second row of the embedding table, corresponding to the second index; and
calculate a first embedded vector based on the first row and the second row.

### Claim 20 (depends on 19)

The system of claim 19, wherein the means for processing is further configured to:
receive a gradient vector; and
update a row of the embedding table based on the gradient vector.
