Neural network model splitting method, apparatus, computer device and storage medium
AssigneeShanghai Cambricon Information Technology Co., Ltd.
InventorsYusong ZHOU, Xiao ZHANG, Linyang WU, Yehao YU, Yunlong XU
Life of this patent
Estimating the term…
Abstract
The present disclosure provides a neural network model splitting method and related products. The scheme provided by the present disclosure splits an operator into a plurality of smaller-scale sub-operators, so that a compute library under a single-core architecture can be called directly, which helps to avoid the extra work caused by re-implementation.
Claims21
1-15. (canceled)
A neural network model splitting method, comprising:
- determining a splitting state set of tensor data associated with an operator of a target layer in a neural network model, wherein the target layer includes at least one layer in the neural network model;
- traversing the splitting state set according to a directed acyclic graph of the neural network model, and determining a state path between each pair of adjacent splitting state sets and a weight of the state path, wherein the state path is indicative of a splitting method of the operator, each state in the splitting state set corresponds to a set of sub-tensor data, and the tensor data is a union of sub-tensor data corresponding to the respective states in the splitting state set;
- determining a target splitting path for the target layer according to the weights of the state paths;
- and
- splitting the operator of the target layer in the neural network model using the target splitting path.
The neural network model splitting method of claim 1, wherein the weight of each state path is determined according to the type and scale of the operator and hardware parameters of a multi-core processor.
A neural network model splitting hardware device, comprising:
- a memory configured to store a computer program;
- and
- a processor, when executing the computer program stored in the memory, configured to:
- determine a splitting state set of tensor data associated with an operator of a target layer in a neural network model, wherein the target layer includes at least one layer in the neural network model;
- traverse the splitting state set according to a directed acyclic graph of the neural network model, and determining a state path between each pair of adjacent splitting state sets and a weight of the state path, wherein the state path is indicative of a splitting method of the operator, each state in the splitting state set corresponds to a set of sub-tensor data, and the tensor data is a union of sub-tensor data corresponding to the respective states in the splitting state set;
- determine a target splitting path for the target layer according to the weights of the state paths;
- and
- split the operator of the target layer in the neural network model using the target splitting path.
Anon-transitory computer readable storage medium, having a computer program stored thereon, the computer program, when executed by a processor, performing a neural network model splitting method, comprising:
- determining a splitting state set of tensor data associated with an operator of a target layer in a neural network model, wherein the target layer includes at least one layer in the neural network model;
- traversing the splitting state set according to a directed acyclic graph of the neural network model, and determining a state path between each pair of adjacent splitting state sets and a weight of the state path, wherein the state path is indicative of a splitting method of the operator, each state in the splitting state set corresponds to a set of sub-tensor data, and the tensor data is a union of sub-tensor data corresponding to the respective states in the splitting state set;
- determining a target splitting path for the target layer according to the weights of the state paths;
- and
- splitting the operator of the target layer in the neural network model using the target splitting path.
Dependencies are inferred from the claim text.