Allocation of machine learning tasks into a shared cache
AssigneeApple Inc.
InventorsFabian P. Wanner, Cecile M. Foret, Xiaozhong Yao, Sundararaman Hariharasubramanian
Life of this patent
Estimating the term…
Abstract
The subject technology receives code corresponding to a neural network (NN) model, the code including particular operations that are performed by the NN model. The subject technology determines, among the particular operations, a set of operations that are to be allocated to a cache of the electronic device that is to execute the NN model. The subject technology generates a set of cache indicators corresponding to the determined set of operations. The subject technology compiles the code and the generated set of cache indicators to provide a compiled binary for the NN model to execute on a target device.
Claims20
A method comprising:
- receiving code corresponding to a neural network (NN) model, the code including particular operations that are performed by the NN model, wherein at least some of the particular operations include respective data that is to be stored in memory of an electronic device during execution of the NN model;
- determining, among the particular operations, a set of operations that are to be allocated to a cache of the electronic device that is to execute the NN model;
- generating a set of cache indicators corresponding to the determined set of operations, wherein the set of cache indicators includes information indicating whether to request an allocation of memory in the cache;
- and
- compiling the code and the generated set of cache indicators to provide a compiled binary for the NN model to execute on a target device.
A system comprising;
- a processor;
- a memory device containing instructions, which when executed by the processor cause the processor to:
- receive code corresponding to a neural network (NN) model, the code including particular operations that are performed by the NN model, wherein at least some of the particular operations include respective data that is to be stored in memory of an electronic device during execution of the NN model;
- determine, among the particular operations, a set of operations that are to be allocated to a cache of the electronic device that is to execute the NN model;
- generate a set of cache indicators corresponding to the determined set of operations, wherein the set of cache indicators includes information indicating whether to request an allocation of memory in the cache;
- and
- compile the code and the generated set of cache indicators to provide a compiled binary for the NN model to execute on a target device.
A non-transitory computer-readable medium comprising instructions, which when executed by a computing device, cause the computing device to perform operations comprising:
- receiving a request for performing an operation by a neural network model, the request including a cache indicator with information indicating whether the operation is to include an allocation of memory in a cache provided by the computing device;
- determining that a request for the allocation of memory in the cache is to be made based at least in part on the cache indicator and the operation;
- and
- sending the request for the allocation of memory to a cache engine to complete the allocation of memory in the cache.
Dependencies are inferred from the claim text.