Method, apparatus and system for multi-module scheduling

A method for multi-module scheduling, comprising:

• receiving, by a computing module in a slave process in a current computing period, a call request of Remote Process Call (RPC) from a master process, wherein the slave process comprises a plurality of computing modules grouped in accordance with a computation direction, and wherein for the computing module having a plurality of pieces of input data, the call request is transmitted by the master process when determining that all of the plurality of pieces of input data to the computing module are ready;
• performing, by the computing module, processing in response to the call request;
• and
• feeding, by the computing module, a response back to the master process when the processing has completed.

An apparatus for multi-module scheduling, comprising a processor and at least one memory storing at least one machine executable instruction, the processor being operative to execute the at least one machine executable instruction to:

• read a pre-stored configuration file storing a directed computation graph associated with a computing task, the computing task comprising a plurality of slave processes each comprising a plurality of computing modules grouped in accordance with a computation direction, the directed computation graph comprising a plurality of nodes each corresponding to one computing module in one slave process, at least two of the nodes having a connecting edge therebetween, an incoming connecting edge of a node being an input edge and an outgoing connecting edge of a node being an output edge;
• initialize, by a master process, states of the nodes and connecting edges in a current computing period;
• determine a node to be called based on the computation direction of the directed computation graph and the states of the nodes, the node to be called comprising a node having all of its input edges in a complete state;
• transmit, to the computing module in the slave process corresponding to the node to be called, a call request of Remote Process Call (RPC) to execute the computing module;
• update the state of the node and the state of each output edge of the node upon receiving a response to the call request;
• and
• proceed with a next computing period after determining that the states of all the nodes in the directed computation graph have been updated.

An apparatus for multi-module scheduling, comprising a processor and at least one memory storing at least one machine executable instruction, the processor being operative to execute the at least one machine executable instruction to:

• receive, in a current computing period, a call request of Remote Process Call (RPC) from a master process, wherein for the apparatus for multi-module scheduling having a plurality of pieces of input data, the call request is transmitted by the master process when determining that all of the plurality of pieces of input data to the apparatus for multi-module scheduling are ready;
• perform processing in response to the call request;
• and
• feed a response back to the master process when the processing has completed.

A system for multi-module scheduling, comprising a master process and a plurality of slave processes each comprising a plurality of computing modules grouped in accordance with a computation direction, wherein

• the master process is configured to read a pre-stored configuration file storing a directed computation graph associated with a computing task, the computing task comprising a plurality of slave processes each comprising a plurality of computing modules grouped in accordance with a computation direction, the directed computation graph comprising a plurality of nodes each corresponding to one computing module in one slave process, at least two of the nodes having a connecting edge therebetween, an incoming connecting edge of a node being an input edge and an outgoing connecting edge of a node being an output edge;
• initialize states of the nodes and connecting edges in a current computing period;
• determine a node to be called based on the computation direction of the directed computation graph and the states of the nodes, the node to be called comprising a node having all of its input edges in a complete state;
• transmit, to the computing module in the slave process corresponding to the node to be called, a call request of Remote Process Call (RPC) to execute the computing module;
• update the state of the node and the state of each output edge of the node upon receiving a response to the call request;
• and proceed with a next computing period after determining that the states of all the nodes in the directed computation graph have been updated, and
• one computing module in the slave process is configured to receive the call request of RPC from the master process, perform processing in response to the call request;
• and feed a response back to the master process when the processing has completed.