Apparatus and method for managing giant model
AssigneeElectronics and Telecommunications Research Institute
InventorsMyung-Hoon CHA, Ki-Dong KANG, Hong-Yeon KIM, Baik-Song AN
Life of this patent
Estimating the term…
Abstract
Disclosed herein is an apparatus and method for managing a giant model. The apparatus includes memory in which at least one program is recorded and a processor for executing the program. The program may perform lightweighting a first model into a second model in consideration of hardware resources, generating partitioning information of the first model based on a result of analysis of the second model, and performing training or inference for the first model based on the generated partitioning information.
Claims20
An apparatus for managing a giant model, comprising:
- memory in which at least one program is recorded;
- and
- a processor for executing the program,
- wherein the program performs
- lightweighting a first model into a second model in consideration of hardware resources,
- generating partitioning information of the first model based on a result of analysis of the second model, and
- performing training or inference for the first model based on the generated partitioning information.
A method for managing a giant model, comprising:
- lightweighting a first model into a second model in consideration of hardware resources;
- generating partitioning information of the first model based on a result of analysis of the second model;
- and
- performing training or inference for the first model based on the generated partitioning information.
A method for managing a giant model, comprising:
- generating a second model by respectively lightweighting multiple modules constituting a first model;
- generating a model component map in which a location of each of the multiple modules constituting the first model is mapped to a location of each of the lightweighted multiple modules constituting the second model;
- analyzing an instance of the second model by loading the instance into memory;
- splitting the first model into multiple partitions in consideration of hardware resources to be occupied by the multiple modules constituting the first model mapped to the loaded second model with reference to the model component map;
- performing training or inference by respectively allocating the multiple partitions split from the first model to multiple servers;
- monitoring the hardware resources when the training or the inference is performed;
- and
- determining whether to again perform model splitting based on a monitoring result.
Dependencies are inferred from the claim text.