Systems and methods of memory allocation for neural networks

A non-transitory computer readable medium comprising instructions, which when executed by at least one processor, configure the processor to perform operations comprising:

• accessing a data processing architecture associated with a neural network to determine dependencies between intermediate data layers of the neural network, the intermediate data layers comprising interconnected nodes of the neural network;
• obtaining dimensions of the intermediate data layers in the neural network;
• calculating a minimum number of data storage portions for executing the neural network based on the dependencies, by:
• segmenting the neural network into time-based processing slices;
• for each time-based processing slice, identifying a number of intermediate data layers needed during execution of the neural network at the time-based processing slice;
• and
• defining the minimum number of data storage portions to equal a largest number of the identified numbers of intermediate data layers needed during execution of the neural network at the time-based processing slices, wherein each data storage portion comprises a memory location configured to store, during execution of the neural network, at least one of the intermediate data layers based on the dependencies;
• assigning each of the intermediate data layers to one of the minimum number of data storage portions;
• determining a memory allocation size for each respective data storage portion of the data storage portions based on the dimensions and dependencies;
• allocating memory on a storage device for each data storage portion in accordance with its respective determined memory allocation size.

A system comprising:

• a storage device;
• at least one processor, comprising instructions, which when executed by the at least one processor, configure the processor to perform operations comprising:
• accessing a data processing architecture associated with a neural network to determine dependencies between intermediate data layers of the neural network;
• obtaining dimensions of the intermediate data layers in the neural network, the intermediate data layers comprising interconnected nodes of the neural network;
• calculating a minimum number of data storage portions for executing the neural network based on the dependencies, by:
• segmenting the neural network into time-based processing slices;
• for each time-based processing slice, identifying a number of intermediate data layers needed during execution of the neural network at the time-based processing slice;
• and
• defining the minimum number of data storage portions to equal a largest number of the identified numbers of intermediate data layers needed during execution of the neural network at the time-based processing slices, wherein each data storage portion comprises a memory location configured to store, during execution of the neural network, at least one of the intermediate data layers based on the dependencies;
• assigning each of the intermediate data layers to one of the minimum number of data storage portions;
• determining a memory allocation size for each respective data storage portion of the data storage portions based on the dimensions and dependencies;
• allocating memory on the storage device for each data storage portion in accordance with its respective determined memory allocation size.