# US 2025/0148362 A1 · Apparatus and method for managing giant model

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0148362 A1
- Document ID: US20250148362A1
- Status: Published application
- Kind: A1
- Application number: 18/678,634
- Filed: 2024-05-30
- Published: 2025-05-08
- Earliest claimed: 2023-11-07
- Assignees: Electronics and Telecommunications Research Institute
- Applicants: Electronics and Telecommunications Research Institute
- Inventors: Myung-Hoon CHA; Ki-Dong KANG; Hong-Yeon KIM; Baik-Song AN
- Web page: https://patentgrep.com/patent/US20250148362A1

## CPC

- G06N 20/00: Machine learning
- G06N 3/063: using electronic means
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Disclosed herein is an apparatus and method for managing a giant model. The apparatus includes memory in which at least one program is recorded and a processor for executing the program. The program may perform lightweighting a first model into a second model in consideration of hardware resources, generating partitioning information of the first model based on a result of analysis of the second model, and performing training or inference for the first model based on the generated partitioning information.

## Claims

### Claim 1

An apparatus for managing a giant model, comprising:
memory in which at least one program is recorded; and
a processor for executing the program,
wherein the program performs
lightweighting a first model into a second model in consideration of hardware resources,
generating partitioning information of the first model based on a result of analysis of the second model, and
performing training or inference for the first model based on the generated partitioning information.

### Claim 2 (depends on 1)

The apparatus of claim 1, wherein, when lightweighting the first model, the program performs
generating the second model by respectively lightweighting multiple modules constituting the first model, and
generating a model component map in which a location of each of the multiple modules constituting the first model is mapped to a location of each of the lightweighted multiple modules constituting the second model.

### Claim 3 (depends on 2)

The apparatus of claim 2, wherein the lightweighted multiple modules constituting the second model are substitutes for the multiple modules constituting the first model at a model definition file level.

### Claim 4 (depends on 2)

The apparatus of claim 2, wherein, when generating the partitioning information, the program performs
analyzing an instance of the second model by loading the instance into memory, and
splitting the first model into multiple partitions in consideration of hardware resources to be occupied by the multiple modules constituting the first model mapped to the loaded second model with reference to the model component map.

### Claim 5 (depends on 4)

The apparatus of claim 4, wherein, when performing the training or the inference, the program performs
respectively allocating the multiple partitions split from the first model to multiple servers to perform the training or the inference.

### Claim 6 (depends on 5)

The apparatus of claim 5, wherein
when performing the training or the inference, the program further performs changing the model component map to correspond to each of the multiple partitions split from the first model, and
the multiple servers perform the training or the inference based on the changed model component map.

### Claim 7 (depends on 6)

The apparatus of claim 6, wherein, in the model component map,
a location of a module of the second model is changed to a location of a module of the first model corresponding thereto when the module of the second model is included in the partition, and
locations of remaining modules of the second model are removed.

### Claim 8 (depends on 6)

The apparatus of claim 6, wherein the program further performs
monitoring the hardware resources when the training or the inference is performed, and
determining whether to again perform model splitting based on a monitoring result.

### Claim 9 (depends on 8)

The apparatus of claim 8, wherein, when it is determined to again perform model splitting, the program performs restoring the changed model component map to the original model component map and again performs operations from splitting the first model.

### Claim 10

A method for managing a giant model, comprising:
lightweighting a first model into a second model in consideration of hardware resources;
generating partitioning information of the first model based on a result of analysis of the second model; and
performing training or inference for the first model based on the generated partitioning information.

### Claim 11 (depends on 10)

The method of claim 10, wherein lightweighting the first model includes
generating the second model by respectively lightweighting multiple modules constituting the first model, and
generating a model component map in which a location of each of the multiple modules constituting the first model is mapped to a location of each of the lightweighted multiple modules constituting the second model.

### Claim 12 (depends on 11)

The method of claim 11, wherein the lightweighted multiple modules constituting the second model are substitutes for the multiple modules constituting the first model at a model definition file level.

### Claim 13 (depends on 11)

The method of claim 11, wherein generating the partitioning information includes
analyzing an instance of the second model by loading the instance into memory, and
splitting the first model into multiple partitions in consideration of hardware resources to be occupied by the multiple modules constituting the first model mapped to the loaded second model with reference to the model component map.

### Claim 14 (depends on 13)

The method of claim 13, wherein performing the training or the inference includes
respectively allocating the multiple partitions split from the first model to multiple servers to perform the training or the inference.

### Claim 15 (depends on 14)

The method of claim 14, wherein:
performing the training or the inference further includes changing the model component map to correspond to each of the multiple partitions split from the first model, and
the multiple servers perform the training or the inference based on the changed model component map.

### Claim 16 (depends on 15)

The method of claim 15, wherein, in the model component map,
a location of a module of the second model is changed to a location of a module of the first model corresponding thereto when the module of the second model is included in the partition, and
locations of remaining modules of the second model are removed.

### Claim 17 (depends on 15)

The method of claim 15, further comprising:
monitoring the hardware resources when the training or the inference is performed, and
determining whether to again perform model splitting based on a monitoring result.

### Claim 18 (depends on 17)

The method of claim 17, wherein
when it is determined to again perform model splitting, restoring the changed model component map to the original model component map is performed, and the method is performed again from splitting the first model.

### Claim 19

A method for managing a giant model, comprising:
generating a second model by respectively lightweighting multiple modules constituting a first model;
generating a model component map in which a location of each of the multiple modules constituting the first model is mapped to a location of each of the lightweighted multiple modules constituting the second model;
analyzing an instance of the second model by loading the instance into memory;
splitting the first model into multiple partitions in consideration of hardware resources to be occupied by the multiple modules constituting the first model mapped to the loaded second model with reference to the model component map;
performing training or inference by respectively allocating the multiple partitions split from the first model to multiple servers;
monitoring the hardware resources when the training or the inference is performed; and
determining whether to again perform model splitting based on a monitoring result.

### Claim 20 (depends on 19)

The method of claim 19, further comprising;
before performing the training or the inference, changing the model component map to correspond to each of the multiple partitions split from the first model,
wherein
in the model component map, a location of a module of the second model is changed to a location of a module of the first model corresponding thereto when the module of the second model is included in the partition, and locations of remaining modules of the second model are removed, and
when it is determined to again perform model splitting, restoring the changed model component map to the original model component map is performed, and then the method is performed again from splitting the first model.
