Stream-based accelerator processing of computational graphs

A computer program product encoded on one or more non-transitory computer storage media, the computer program product comprising instructions that, when executed by a system comprising a plurality of hardware streams, cause the system to perform program operations comprising:

• for each of a plurality of operations represented in a computational graph, assigning the operation to a respective stream in the plurality of streams of the system, each stream configured to queue operations assigned to the stream and to execute the queued operations in a defined order on a respective hardware resource for the stream;
• configuring a first stream of the plurality of streams to stall performance of a first operation assigned to the first stream until all inputs to the first operation have been computed, wherein a first input to the first operation comprises an output of a second operation assigned to a second, different stream of the plurality of streams;
• and
• performing, by each stream in the plurality of streams, the operations that were assigned to the stream in a defined order, including performing at least one operation by the first stream in parallel with at least one operation by the second stream.

A method performed by a system comprising a plurality of hardware streams, comprising:

• for each of a plurality of operations represented in a computational graph, assigning the operation to a respective stream in the plurality of streams of the system, each stream configured to queue operations assigned to the stream and to execute the queued operations in a defined order on a respective hardware resource for the stream;
• configuring a first stream of the plurality of streams to stall performance of a first operation assigned to the first stream until all inputs to the first operation have been computed, wherein a first input to the first operation comprises an output of a second operation assigned to a second, different stream of the plurality of streams;
• and
• performing, by each stream in the plurality of streams, the operations that were assigned to the stream in a defined order, including performing at least one operation by the first stream in parallel with at least one operation by the second stream.

A system comprising:

• a plurality of hardware streams;
• and
• one or more non-transitory computer storage media encoded with instructions that, when executed, cause performance of operations comprising:
• for each of a plurality of operations represented in a computational graph, assigning the operation to a respective stream in the plurality of streams of the system, each stream configured to queue operations assigned to the stream and to execute the queued operations in a defined order on a respective hardware resource for the stream;
• configuring a first stream of the plurality of streams to stall performance of a first operation assigned to the first stream until all inputs to the first operation have been computed, wherein a first input to the first operation comprises an output of a second operation assigned to a second, different stream of the plurality of streams;
• and
• performing, by each stream in the plurality of streams, the operations that were assigned to the stream in a defined order, including performing at least one operation by the first stream in parallel with at least one operation by the second stream.