Deep Neural Networks Compiler for a Trace-Based Accelerator

A compiler system for compiling neural network code to a plurality of executable instructions or other into executable code for execution by a computational acceleration system having a first memory circuit and one or more acceleration circuits having a maps data buffer and a kernel data buffer, the compiler system comprising:

• a second memory circuit storing the neural network code;
• and
• one or more processors coupled to the second memory circuit, the one or more processors adapted to generate a list of neural network layer model objects from the neural network code;
• to fuse available functions and layers in the list of neural network layer model objects;
• to select a cooperative mode, an independent mode, or a combined cooperative and independent mode for code execution by the one or more acceleration circuits;
• to select a data movement mode and an ordering of computations which reduces usage of the first memory circuit;
• to generate an ordered sequence of load objects, compute objects, and store objects;
• and to convert the ordered sequence of load objects, compute objects, and store objects into the plurality of executable instructions.

A method of compiling neural network code to a plurality of executable instructions or other into executable code for execution by a computational acceleration system having a memory circuit and one or more acceleration circuits having a maps data buffer and a kernel data buffer, the method comprising:

• using one or more processors, from the neural network code, generating a list of neural network layer model objects;
• using the one or more processors, fusing available functions and layers in the list of neural network layer model objects;
• using the one or more processors, selecting a cooperative mode, an independent mode, or a combined cooperative and independent mode for code execution by the one or more acceleration circuits;
• using the one or more processors, determining memory circuit usage when kernel data is maintained in the kernel data buffer for a predetermined number of computations with reordering of maps data movement and computations to utilize the selected kernel data which is currently loaded in the kernel data buffer;
• using the one or more processors, determining memory circuit usage when maps data is maintained in the maps data buffer for a predetermined number of computations with reordering of kernel data movement and computations to utilize the selected maps data which is currently loaded in the maps data buffer;
• using the one or more processors, determining memory circuit usage when maps data movement is reordered, kernel data movement is reordered, and computations are reordered to utilize one or more rows of the selected maps data or kernel data which is currently loaded respectively in the maps data buffer or in the kernel data buffer;
• using the one or more processors, reordering at least one computation to utilize one or more rows of selected maps data or kernel data which is currently loaded respectively in the maps data buffer or in the kernel data buffer;
• using the one or more processors, selecting a data movement mode and an ordering of computations which minimizes usage of the memory circuit;
• using the one or more processors, generating an ordered sequence of load objects, compute objects, and store objects;
• further ordering the sequence of load objects, compute objects, and store objects to reduce any computational latency and to provide data pre-fetching;
• and
• using the one or more processors, converting the ordered sequence of load objects, compute objects, and store objects into the plurality of executable instructions.