Method and Device for Memory Scheduling for Code Generation for a Program Code for Computing an Artificial Neural Network in a Hardware Environment

Abstract

A computer-implemented method for performing memory scheduling for code generation to determine a code for computing a neural network in a hardware environment includes (i) providing successive computation steps of the neural network, wherein a plurality of the computation steps provide for the computation of a weight prefetching layer for which weight prefetching is applicable, (ii) assigning possible weight prefetching to at least a portion of the plurality of weight-prefetching layers in order to obtain multiple different combinations of weight-prefetching layers, wherein the weight prefetching provides for preloading of network parameters into a working memory of the hardware environment for the respective weight-prefetching layer, (iii) performing memory scheduling for the code to be generated for the plurality of different combinations, wherein the respective memory scheduling is carried out for the successive computation steps of the neural network while taking into account the respective combination of weight-prefetching layers for which weight prefetching is provided, wherein for each memory scheduling, for every computation step, the memory region of the respective input data block, output data block, and at least one network parameter block in the working memory is defined, and a maximum required memory space of the working memory is ascertained, and (iv) selecting one of the memory schedulings depending on the maximum required memory space of the working memory.

Dependencies are inferred from the claim text.