# US 2025/0165804 A1 · Method and apparatus with prediction of training time of neural network model

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0165804 A1
- Document ID: US20250165804A1
- Status: Published application
- Kind: A1
- Application number: 18/945,888
- Filed: 2024-11-13
- Published: 2025-05-22
- Earliest claimed: 2023-11-22
- Assignees: Samsung Electronics Co., Ltd.; Korea Advanced Institute of Science and Technology
- Applicants: Samsung Electronics Co., Ltd.; Korea Advanced Institute of Science and Technology
- Inventors: Minsoo Rhu; Jehyeon Bang
- Web page: https://patentgrep.com/patent/US20250165804A1

## CPC

- G06N 3/098: Distributed learning, e.g. federated learning
- G06N 3/042: Knowledge-based neural networks; Logical representations of neural networks
- G06N 3/045: Combinations of networks
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Disclosed is a method and apparatus for predicting a training time. The method includes generating a first execution graph representing a distributed learning process of a neural network model, wherein the first execution graph includes a computation node representing a repeated computation of the neural network model based on a parallelism technique; generating a second execution graph based on the first execution graph and a target operator corresponding to the computation node, wherein the second execution graph includes an execution time of a hardware kernel performing the target operator; and predicting a training time of the neural network model based on the second execution graph.

## Claims

### Claim 1

A method of predicting a training time, comprising:
generating a first execution graph representing a distributed learning process of a neural network model, wherein the first execution graph includes a computation node representing a repeated computation of the neural network model based on a parallelism technique;
generating a second execution graph based on the first execution graph and a target operator corresponding to the computation node, wherein the second execution graph includes an execution time of a hardware kernel performing the target operator; and
predicting a training time of the neural network model based on the second execution graph.

### Claim 2 (depends on 1)

The method of claim 1, wherein the first execution graph comprises a plurality of computation nodes and a plurality of communication nodes for the distributed learning process.

### Claim 3 (depends on 2)

The method of claim 2, wherein the generating of the first execution graph comprises:
dividing a computation task of the neural network model based on the parallelism technique to obtain the plurality of computation nodes.

### Claim 4 (depends on 2)

The method of claim 2, wherein the generating of the first execution graph comprises:
identifying a communication pattern based on the parallelism technique; and
deriving a dependency among the plurality of computation nodes and the plurality of communication nodes based on the communication pattern, wherein an edge of the first execution graph represents the dependency.

### Claim 5 (depends on 1)

The method of claim 1, wherein the first execution graph is based on a plurality of independent dimensions corresponding to a plurality of parallelism techniques, respectively.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
determining a communication pattern based on a structure of the neural network model, the parallelism technique and a configuration of a hardware device to be used in the distributed learning process, wherein the first execution graphs is based on the communication patterns.

### Claim 7 (depends on 5)

The method of claim 5, further comprising:
determining a dependency between one or more computation nodes and one or more communication nodes based on a time point at which a communication task occurs in the communication pattern, wherein the first execution graph includes an edge representing the dependency.

### Claim 8 (depends on 1)

The method of claim 1, wherein the predicting of the training time comprises:
selecting the target operator to be profiled; and
predicting an execution time of the target operator by a profiling module.

### Claim 9 (depends on 8)

The method of claim 8, wherein the selecting of the target operator comprises:
selecting a repeated layer in the neural network model;
dividing computations in the repeated layer according to the parallelism technique to obtain the repeated computation; and
selecting the target operator based on the repeated computation.

### Claim 10 (depends on 1)

The method of claim 1, further comprising:
profiling the target operator by a profiling module to obtain an execution time; and
predicting the training time based on the second execution graph and the execution time of the target operator.

### Claim 11 (depends on 1)

The method of claim 1, wherein the generating of the second execution graph comprises:
collecting information about an execution time and a hardware type of the hardware kernel by profiling the target operator; and
converting the first execution graph into the second execution graph by replacing the computation node of the first execution graph with a node representing the hardware kernel based on the collected information.

### Claim 12 (depends on 11)

The method of claim 11, wherein the training is predicted based on collected information.

### Claim 13 (depends on 1)

The method of claim 1, wherein the parallelism technique comprise at least one of:
a data parallelism technique, a tensor parallelism technique, or a pipeline parallelism technique.

### Claim 14 (depends on 1)

The method of claim 1, wherein the neural network model comprises a transformer-based large-scale language model having a plurality of layers with a same hyperparameter,
wherein each of the plurality of layers is configured to perform computations divided by a predetermined size in the distributed learning process.

### Claim 15 (depends on 1)

A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform the method of claim 1.

### Claim 16

An apparatus for predicting a training time, comprising:
a memory storing a neural network; and
at least one processor configured to: generate a first execution graph representing a distributed learning process of the neural network model, wherein the first execution graph includes a computation node representing a repeated computation of the neural network model based on a parallelism technique; generate a second execution graph based on the first execution graph and a target operator corresponding to the computation node, wherein the second execution graph includes an execution time of a hardware kernel performing the target operator; and predict a training time of the neural network model based on the second execution graph.

### Claim 17 (depends on 16)

The apparatus of claim 16, wherein the first execution graph comprises a plurality of computation nodes and a plurality of communication nodes for the distributed learning process.

### Claim 18 (depends on 16)

The apparatus of claim 16, wherein the at least one processor is configured to:
divide a computation task of the neural network model based on the parallelism technique to obtain the plurality of computation nodes.

### Claim 19 (depends on 16)

The apparatus of claim 16, wherein the at least one processor is configured to:
select the target operator to be profiled; and
predict an execution time of the target operator by a profiling module.

### Claim 20 (depends on 19)

The apparatus of claim 19, wherein the at least one processor is configured to:
select a repeated layer in the neural network model;
divide computations in the repeated layer according to the parallelism technique to obtain the repeated computation; and
select the target operator based on the repeated computation.

### Claim 21

A method comprising:
obtaining information representing a structure of a neural network model;
dividing an operation of the neural network model into a plurality of repeated computations based on a parallelism technique and a hardware device to obtain a plurality of computation nodes;
generating a first execution graph including the plurality of computation nodes based on the structure of the neural network model and the hardware device;
generating a second execution graph based on the first execution graph and a hardware kernel of the hardware device, wherein the second execution graph includes a node representing the hardware kernel;
profiling a target operator of the plurality of computation nodes based on the hardware kernel to obtain an execution time; and
computing a predicted training time based on the second execution graph and the execution time.
