Software defined redundant allocation safety mechanism in an artificial neural network processor

An apparatus for end to end failure detection for use in a neural network (NN) processor configured to implement a target neural network, comprising:

• a plurality of unallocated redundant hardware resources within said neural network processor, said redundant hardware resources including compute, memory, routing, and control elements;
• a main computational path allocated during an offline compilation process via a compiler from said plurality of unallocated redundant hardware resources, said main computational path to be protected from end to end failures;
• one or more redundant computational paths allocated during said offline compilation process via said compiler from said plurality of unallocated redundant hardware resources different from said plurality of unallocated redundant hardware resources allocated to said main computational path, where said compiler ensures that said main computational path and said one or more redundant computational paths go through and use different resources with no overlap of compute, memory, routing, and control hardware resource elements, wherein said main computational path and said one or more redundant computational paths each function to perform the same calculations for the same target neural network, said one or more redundant computational paths operative to protect said main computational path from end to end failures whereby the same input data is provided to said main computational path and said one or more redundant computational paths at the same time, and whereby whole or partial layers of a whole or partial network are allocated;
• one or more cyclic redundancy code (CRC) engines operative to generate CRC checksums on tensor stream data output from said main computational path and said one or more redundant computational paths, compare said CRC checksums from said main computational path and said one or more redundant computational paths with each other, and to detect an error if said calculated CRC checksums do not match;
• and
• wherein said main computational path and said one or more redundant computational paths use different data resources selected from a group consisting of stream managers (SMs), portions of L4 memory allocated to said stream managers, input buffers (IBs), portions of L3 memory, input aligners (IAs), subclusters (SCs), activation processing units (APUs), and output buffers (OBs).

A method of end to end failure detection for use in a neural network processor configured to implement a target neural network, the method comprising:

• providing a plurality of unallocated redundant hardware resources in said neural network processor, said unallocated redundant hardware resources including compute, memory, routing, and control elements;
• determining a resource allocation scheme in accordance with desired performance goals for a target neural network;
• allocating during an offline compilation process via a compiler a plurality of redundant computational tensor data flow paths from said plurality of unallocated redundant hardware resources, where said compiler ensures that said plurality of redundant computational tensor data flow paths go through and use different resources, said plurality of redundant computational tensor data flow paths functionally identical to each other and operative to provide protection from end to end failures by way of said redundancy wherein each redundant computation tensor data flow path uses different unallocated redundant hardware resources with no overlap of compute, memory, routing, and control hardware resource elements, wherein said plurality of redundant computational tensor data flow paths each function to perform the same calculations for the same target neural network, whereby whole or partial layers of a whole or partial network are allocated;
• providing the same input data to each of said redundant computational tensor data flow paths at the same time;
• comparing cyclic redundancy code (CRC) checksums generated for each redundant computation tensor data flow path and detecting an error if a mismatch is detected;
• and
• wherein said plurality of redundant computational tensor data flow paths use different data resources selected from a group consisting of stream managers (SMs), portions of L4 memory allocated to said stream managers, input buffers (IBs), portions of L3 memory, input aligners (IAs), subclusters (SCs), activation processing units (APUs), and output buffers (OBs).