Optimizing data partitioning and replacement strategy for convolutional neural networks

A machine-readable non-transitory medium having stored thereon machine-executable instructions for optimizing a convolutional layer of a convolutional neural network (CNN), the instructions comprising:

• for each of a plurality of internal memory replacement schemes, wherein each internal memory replacement scheme identifies one of input feature map (IFM) data, kernel data, or output feature map (OFM) data to maintain in an internal memory until all calculations requiring said one of the IFM data, kernel data, or OFM data for the convolutional layer are completed, identify optimal data partition sizes of the IFM data, the kernel data, and the OFM data which minimizes a computation cost for the convolutional layer given each corresponding internal memory replacement scheme;
• determining a combination of a selected one of the plurality of internal memory replacement schemes with corresponding optimal partition sizes of the IFM data, the kernel data, and the OFM data having a lowest computation cost among all combinations of the plurality of internal memory replacement schemes and corresponding optimal partition sizes;
• and
• implementing the CNN to use the selected one of the plurality of internal memory replacement schemes with the corresponding optimal partition sizes of the IFM data, the kernel data, and the OFM data for calculation of the convolutional layer.

A computer implemented method for optimizing an internal memory for calculation of a convolutional layer of a convolutional neural network (CNN), the method comprising:

• determining a computation cost of calculating the convolutional layer using each combination of a memory management scheme of a plurality of memory management schemes and data partition sizes of input feature map (IFM) data, kernel data, and output feature map (OFM) data to be loaded in the internal memory;
• identifying one combination of a memory management scheme and data partition sizes having a lowest computation cost for the convolutional layer;
• and
• implementing the CNN to use the one combination for calculation of the convolutional layer.