Model fusion method, electronic device and storage medium
AssigneeBeijing Baidu Netcom Science Technology Co., Ltd.
InventorsZiyang Xu, Xuyi Chen, Bo Ke, Zhengjie Huang, Shikun Feng, Shiwei Huang
Life of this patent
Estimating the term…
Abstract
A model fusion method includes: calling a main process during a pre-training process of a large model, to cache intermediate model parameters obtained during the pre-training process into a main buffer; and calling a sub-process via the main process to read the intermediate model parameters from the main buffer and perform a parameter fusion process based on the intermediate model parameters.
Claims20
A model fusion method, comprising:
- calling a main process, during a pre-training process of a large model, to cache intermediate model parameters obtained during the pre-training process into a main buffer;
- and
- calling a sub-process via the main process to read the intermediate model parameters from the main buffer and perform a parameter fusion process based on the intermediate model parameters.
An electronic device, comprising:
- at least one processor;
- and
- a memory communicatively connected to the at least one processor;
- wherein the memory stores an instruction executable by the at least one processor, and the instruction, when being executed by the at least one processor, enables the at least one processor to:
- call a main process, during a pre-training process of a large model, to cache intermediate model parameters obtained during the pre-training process into a main buffer;
- and
- call a sub-process via the main process to read the intermediate model parameters from the main buffer and perform a parameter fusion process based on the intermediate model parameters.
A non-transitory computer-readable storage medium having a computer instruction stored thereon, wherein the computer instruction is used to cause a computer to implement a method comprising:
- calling a main process, during a pre-training process of a large model, to cache intermediate model parameters obtained during the pre-training process into a main buffer;
- and
- calling a sub-process via the main process to read the intermediate model parameters from the main buffer and perform a parameter fusion process based on the intermediate model parameters.
Dependencies are inferred from the claim text.