US 12,547,453 B2Granted

Graph streaming neural network processing system and method thereof

AssigneeBlaize, Inc.

InventorsVenkata Ganapathi Puppala, Val G. Cook, Srinivasulu Nagisetty

Life of this patent

Estimating the term…

Loading patent…

Abstract

Disclosed herein is a graph streaming neural network processing system comprising a first processor array, a second processor, and a thread scheduler. The thread scheduler dispatches a thread of a first node to the first processor array or the second processor, wherein the thread is executed to generate output data comprising a data unit stored in a private data buffer of the second processor. The thread scheduler determines that the data unit is sufficient for executing a thread of a second node. The second node is dependent on the output data generated by execution of a plurality of threads of the first node. Upon determining that the data unit is sufficient, the thread scheduler dispatches the thread of the second node. The thread scheduler determines to dispatch a subsequent thread of the first node for execution when a predefined threshold buffer size is available on the private data buffer.

Claims18

Dependent claims

A graph streaming processing system comprising:

A first processor array;
a second processor;
and
a thread scheduler configured to:
dispatch at least one thread associated with a first node, to one of the first processor array and the second processor, to generate an output data comprising at least one data unit, wherein the at least one data unit is stored in a private data buffer of the second processor;
determine sufficiency of the at least one data unit for executing at least one thread of a second node, wherein the second node is identified to be dependent on the output data generated by execution of a plurality of threads of the first node;
dispatch the at least one thread of the second node, to one of the first processor array and the second processor, upon determining that the at least one data unit is sufficient;
and
determine to dispatch at least one subsequent thread of the first node for execution when a predefined threshold buffer size is available on the private data buffer,
wherein the first processor array is configured to:
receive the at least one thread dispatched by the thread scheduler;
retrieve input data required for execution of the at least one thread from a data buffer shared between the first processor array and the second processor;
execute the at least one thread to generate the output data;
and
perform at least one of:
writing the output data to a shared data buffer, upon determination that the subsequent thread, dependent on the at least one thread, is being dispatched to the first processor array, wherein the shared data buffer is a memory subsystem shared by the first processor array and the second processor;
and
writing the output data into the private data buffer, upon determination that the subsequent thread is being dispatched to the second processor.

15 more elements

A method comprising:

dispatching, by a thread scheduler of a graph streaming processing system, one thread associated with a first node, to one of a first processor array and a second processor of the graph streaming processing system, to generate an output data comprising at least one data unit, wherein the at least one data unit is stored in a private data buffer of the second processor;
determining, by the thread scheduler, sufficiency of at least one data unit for executing at least one thread of the second node, wherein the second node is identified to be dependent on the output data generated by execution of a plurality of threads of the first node;
dispatching, by the thread scheduler, the at least one thread of the second node, to one of the first processor array and the second processor, upon determining that the at least one data unit is sufficient;
and
determining, by the thread scheduler, to dispatch at least one subsequent thread of the first node for execution when a predefined threshold buffer size is available on the private data buffer,
wherein the first processor array is configured to:
receive the at least one thread dispatched by the thread scheduler;
retrieve input data required for execution of the at least one thread from a data buffer shared between the first processor array and the second processor;
execute the at least one thread to generate the output data;
and
perform at least one of:
writing the output data to a shared data buffer, upon determination that the subsequent thread, dependent on the at least one thread, is being dispatched to the first processor array, wherein the shared data buffer is a memory subsystem shared by the first processor array and the second processor;
and
writing the output data into the private data buffer, upon determination that the subsequent thread is being dispatched to the second processor.

11 more elements

A non-transitory computer-readable medium having program instructions stored thereon, wherein the program instructions, when executed by a thread-scheduler of a graph streaming processing system, facilitate:

dispatching, by a thread scheduler of a graph streaming processing system, one thread associated with a first node, to one of a first processor array and a second processor of the graph streaming processing system, to generate an output data comprising at least one data unit, wherein the at least one data unit is stored in a private data buffer of the second processor;
determining, by the thread scheduler, sufficiency of at least one data unit for executing at least one thread of the second node, wherein the second node is identified to be dependent on the output data generated by execution of a plurality of threads of the first node;
dispatching, by the thread scheduler, the at least one thread of the second node, to one of the first processor array and the second processor, upon determining that the at least one data unit is sufficient;
and
determining, by the thread scheduler, to dispatch at least one subsequent thread of the first node for execution when a predefined threshold buffer size is available on the private data buffer,
wherein the first processor array is configured to:
receive the at least one thread dispatched by the thread scheduler;
retrieve input data required for execution of the at least one thread from a data buffer shared between the first processor array and the second processor;
execute the at least one thread to generate the output data;
and
perform at least one of:
writing the output data to a shared data buffer, upon determination that the subsequent thread, dependent on the at least one thread, is being dispatched to the first processor array, wherein the shared data buffer is a memory subsystem shared by the first processor array and the second processor;
and
writing the output data into the private data buffer, upon determination that the subsequent thread is being dispatched to the second processor.

11 more elements

Dependencies are inferred from the claim text.

Facts

Application: 18/210,706Filed 16 Jun 2023
Priority: IN 20224103586522 Jun 2022
Published: US 2023/0418666 A128 Dec 2023
Granted: 10 Feb 2026
Term ends: Not estimatedThe source records don’t support an estimate.

Classification3 codes

G06F 9/4881
By program, e.g. task dispatcher, supervisor, operating system: scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
G06F 15/8015
Single instruction multiple data [SIMD] multiprocessors: one dimensional arrays, e.g. rings, linear arrays, buses
G06F
Computing and data processing