# US 2023/0274157 A1 · Ingestion of data for machine learning distributed training

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0274157 A1
- Document ID: US20230274157A1
- Status: Published application
- Kind: A1
- Application number: 18/312,276
- Filed: 2023-05-04
- Published: 2023-08-31
- Earliest claimed: 2023-05-04
- Assignees: Intel Corporation
- Applicants: Intel Corporation
- Inventors: Vikram Saletore; Yaser Afshar
- Web page: https://patentgrep.com/patent/US20230274157A1

## CPC

- G06N 3/063: using electronic means
- G06N 3/098: Distributed learning, e.g. federated learning
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/09: Supervised learning
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Systems and methods include technology that identifies a first shard of shards of training data, where the training data is stored on a storage array, where the training data is divided into shards. The technology stores the first shard onto a first data storage of a first compute node, and copies the first shard from the first data storage of the first compute node onto a second data storage of the first compute node. The technology trains a machine learning model on the first shard stored in the second data storage during a first epoch of a training phase.

## Claims

### Claim 1

A first compute node comprising:
a network controller to communicate with second compute nodes and a storage array, wherein the storage array is to store training data that is divided into shards;
one or more of a processor implemented in one or more of configurable logic or fixed-functionality logic, or an accelerator implemented in one or more of configurable logic or fixed-functionality logic;
a first data storage;
a second data storage; and
a memory including a set of instructions, which when executed by the one or more of the processor or the accelerator, cause the first compute node to:
store a first shard of the shards onto the first data storage;
copy the first shard from the first data storage onto the second data storage; and
train a machine learning model on the first shard stored in the first data storage during a first epoch of a training phase.

### Claim 2 (depends on 1)

The first compute node of claim 1, wherein the set of instructions, which when executed by the one or more of the processor or the accelerator, cause the first compute node to:
receive, during the first epoch, a second portion of a second shard of the shards from at least one of the second compute nodes; and
store the second portion into the second data storage.

### Claim 3 (depends on 2)

The first compute node of claim 2, wherein the set of instructions, which when executed by the one or more of the processor or the accelerator, cause the first compute node to:
transfer the second portion of the second shard from the second data storage to the first data storage.

### Claim 4 (depends on 3)

The first compute node of claim 3, wherein to transfer the second portion of the second shard from the second data storage to the first data storage, the set of instructions, which when executed by the one or more of the processor or the accelerator, cause the first compute node to:
overwrite a first portion of the first shard stored in the first data storage with the second portion to generate a third shard.

### Claim 5 (depends on 4)

The first compute node of claim 4, wherein the set of instructions, which when executed by the one or more of the processor or the accelerator, cause the first compute node to:
shuffle data of the second portion and data of the first shard.

### Claim 6 (depends on 4)

The first compute node of claim 4, wherein the set of instructions, which when executed by the one or more of the processor or the accelerator, cause the first compute node to:
train the machine learning model based on the third shard stored in the first data storage during a second epoch of the training phase.

### Claim 7 (depends on 3)

The first compute node of claim 3, wherein to transfer the second portion of the second shard from the second data storage to the first data storage, the set of instructions, which when executed by the one or more of the processor or the accelerator, cause the first compute node to:
transfer the second portion from the second data storage to the first data storage at memory speed, and
wherein the first data storage is a first memory based local data storage, and
wherein the second data storage is a second memory based local data storage.

### Claim 8

A semiconductor apparatus, the semiconductor apparatus comprising:
one or more substrates; and
logic coupled to the one or more substrates, wherein the logic is implemented in one or more of configurable logic or fixed-functionality logic, the logic coupled to the one or more substrates to:
identify a first shard of shards of training data, wherein the training data is stored on a storage array, wherein the training data is divided into shards;
store the first shard onto a first data storage of a first compute node;
copy the first shard from the first data storage of the first compute node onto a second data storage of the first compute node; and
train a machine learning model on the first shard stored in the first data storage during a first epoch of a training phase.

### Claim 9 (depends on 8)

The apparatus of claim 8, wherein the logic coupled to the one or more substrates is to:
receive, during the first epoch, a second portion of a second shard of the shards from a second compute node; and
store the second portion into the second data storage.

### Claim 10 (depends on 9)

The apparatus of claim 9, wherein the logic coupled to the one or more substrates is to:
transfer the second portion of the second shard from the second data storage to the first data storage.

### Claim 11 (depends on 10)

The apparatus of claim 10, wherein to transfer the second portion of the second shard from the second data storage to the first data storage, the logic coupled to the one or more substrates is to:
overwrite a first portion of the first shard stored in the first data storage with the second portion to generate a third shard.

### Claim 12 (depends on 11)

The apparatus of claim 11, wherein the logic coupled to the one or more substrates is to:
shuffle data of the second portion and data of the first shard.

### Claim 13 (depends on 11)

The apparatus of claim 11, wherein the logic coupled to the one or more substrates is to:
train the machine learning model based on the third shard stored in the first data storage during a second epoch of the training phase.

### Claim 14 (depends on 10)

The apparatus of claim 10, wherein to transfer the second portion of the second shard from the second data storage to the first data storage, the logic coupled to the one or more substrates is to:
transfer the second portion from the second data storage to the first data storage at memory speed,
wherein the first data storage is a first memory based local data storage, and
wherein the second data storage is a second memory based local data storage.

### Claim 15 (depends on 8)

The apparatus of claim 8, wherein the logic coupled to the one or more substrates includes transistor channel regions that are positioned within the one or more substrates.

### Claim 16

A method comprising:
identifying a first shard of shards of training data, wherein the training data is stored on a storage array, wherein the training data is divided into shards;
storing the first shard onto a first data storage of a first compute node;
copying the first shard from the first data storage of the first compute node onto a second data storage of the first compute node; and
training a machine learning model on the first shard stored in the first data storage during a first epoch of a training phase.

### Claim 17 (depends on 16)

The method of claim 16, further comprising:
receiving, during the first epoch, a second portion of a second shard of the shards from a second compute node; and
storing the second portion into the second data storage.

### Claim 18 (depends on 17)

The method of claim 17, further comprising:
transferring the second portion of the second shard from the second data storage to the first data storage.

### Claim 19 (depends on 18)

The method of claim 18, wherein the transferring further comprises:
overwriting a first portion of the first shard stored in the first data storage with the second portion to generate a third shard.

### Claim 20 (depends on 19)

The method of claim 19, further comprising:
shuffling data of the second portion and data of the first shard; and
training the machine learning model based on the third shard stored in the first data storage during a second epoch of the training phase,
wherein the transferring comprises transferring the second portion from the second data storage to the first data storage at memory speed,
wherein the first data storage is a first memory based local data storage, and
wherein the second data storage is a second memory based local data storage.
