Method, system and device for parallel processing of data, and storage medium

A device, comprising:

• one or more processors;
• and
• a memory configured to store one or more programs;
• wherein the one or more programs when executed by the one or more processors cause the one or more processors to perform operations, the operations comprising:
• identifying, from a plurality of first computing nodes, at least three first computing nodes which have a logical relationship, and defining the at least three first computing nodes which have the logical relationship as a first parallel node group, wherein the first parallel node group comprises a first preceding node and at least two first subsequent nodes, and wherein the at least two first subsequent nodes are located downstream of the first preceding node and have a direct logical relationship with the first preceding node;
• acquiring a first input data model of the first preceding node and generating a first input tensor of the first preceding node;
• computing a first output tensor of the first preceding node according to the first input data model and the first input tensor;
• acquiring a second input data model of the at least two first subsequent nodes and using the first output tensor as a second input tensor;
• and
• computing a second output tensor of each first subsequent node respectively according to the second input data model and the second input tensor to obtain a first computation result of the first parallel node group;
• wherein before acquiring the first input data model of the first preceding node and generating the first input tensor of the first preceding node, the operations further comprise:
• confirming a number of reference times of the first preceding node according to a number of the at least two first subsequent nodes;
• wherein after computing the second output tensors of the at least two first subsequent nodes respectively according to the second input data model and the second input tensor to obtain the first computation result of the first parallel node group, the operations further comprise:
• determining whether a computation at each first subsequent node is completed;
• reducing the number of reference times of the first preceding node by one separately after the computation at each first subsequent node is completed;
• determining whether the number of reference times of the first preceding node is zero;
• and
• deleting the first output tensor in an on-chip memory and storing the first output tensor in an off-chip memory in a case where the number of reference times of the first preceding node is zero;
• wherein after deleting the first output tensor in the on-chip memory and storing the first output tensor in the off-chip memory in the case where the number of reference times of the first preceding node is zero, the operations further comprise:
• determining whether a second parallel node group which uses any one of the at least two first subsequent nodes as a second preceding node exists in downstream of the at least two first subsequent nodes of the first parallel node group, wherein the second parallel node group comprises the second preceding node and at least two second subsequent nodes;
• in a case where the second parallel node group which uses any one of the at least two first subsequent nodes as the second preceding node exists in downstream of the at least two first subsequent nodes, acquiring a third input data model of the second preceding node and generating a third input tensor of the second preceding node;
• computing a third output tensor of the second preceding node according to the third input data model and the third input tensor;
• acquiring a fourth input data model of the at least two second subsequent nodes and using the third output tensor as a fourth input tensor;
• and computing fourth output tensors of the at least two second subsequent nodes respectively according to the fourth input data model and the fourth input tensor to obtain a second computation result of the second parallel node group;
• and
• receiving a computation completion instruction to end a computation in a case where the second parallel node group does not exist in downstream of the at least first subsequent nodes.

A non-transitory computer-readable storage medium configured to store computer programs which, when executed by a processor, cause the processor to perform operations, the operations comprising:

• identifying, from a plurality of first computing nodes, at least three first computing nodes which have a logical relationship, and defining the at least three first computing nodes which have the logical relationship as a first parallel node group, wherein the first parallel node group comprises a first preceding node and at least two first subsequent nodes, and wherein the at least two first subsequent nodes are located downstream of the first preceding node and have a direct logical relationship with the first preceding node;
• acquiring a first input data model of the first preceding node and generating a first input tensor of the first preceding node;
• computing a first output tensor of the first preceding node according to the first input data model and the first input tensor;
• acquiring a second input data model of the at least two first subsequent nodes and using the first output tensor as a second input tensor;
• and
• computing a second output tensor of each first subsequent node respectively according to the second input data model and the second input tensor to obtain a first computation result of the first parallel node group;
• wherein before acquiring the first input data model of the first preceding node and generating the first input tensor of the first preceding node, the operations further comprise:
• confirming a number of reference times of the first preceding node according to a number of the at least two first subsequent nodes;
• wherein after computing the second output tensors of the at least two first subsequent nodes respectively according to the second input data model and the second input tensor to obtain the first computation result of the first parallel node group, the operations further comprise:
• determining whether a computation at each first subsequent node is completed:
• reducing the number of reference times of the first preceding node by one separately after the computation at each first subsequent node is completed;
• determining whether the number of reference times of the first preceding node is zero;
• and
• deleting the first output tensor in an on-chip memory and storing the first output tensor in an off-chip memory in a case where the number of reference times of the first preceding node is zero;
• wherein after deleting the first output tensor in the on-chip memory and storing the first output tensor in the off-chip memory in the case where the number of reference times of the first preceding node is zero, the operations further comprise:
• determining whether a second parallel node group which uses any one of the at least two first subsequent nodes as a second preceding node exists in downstream of the at least two first subsequent nodes of the first parallel node group, wherein the second parallel node group comprises the second preceding node and at least two second subsequent nodes;
• in a case where the second parallel node group which uses any one of the at least two first subsequent nodes as the second preceding node exists in downstream of the at least two first subsequent nodes, acquiring a third input data model of the second preceding node and generating a third input tensor of the second preceding node;
• computing a third output tensor of the second preceding node according to the third input data model and the third input tensor;
• acquiring a fourth input data model of the at least two second subsequent nodes and using the third output tensor as a fourth input tensor;
• and computing fourth output tensors of the at least two second subsequent nodes respectively according to the fourth input data model and the fourth input tensor to obtain a second computation result of the second parallel node group;
• and
• receiving a computation completion instruction to end a computation in a case where the second parallel node group does not exist in downstream of the at least first subsequent nodes.