# US 11,907,760 B2 · Systems and methods of memory allocation for neural networks > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,907,760 B2 - Document ID: US11907760B2 - Status: Granted - Kind: B2 - Application number: 15/711,781 - Filed: 2017-09-21 - Published: 2018-03-29 - Granted: 2024-02-20 - Earliest claimed: 2016-09-23 - Assignees: Apple Inc. - Applicants: Apple Inc. - Inventors: Francesco Rossi; Marco Zuliani - Web page: https://patentgrep.com/patent/US11907760B2 ## CPC - G06F 9/5016: {the resource being the memory} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A method may include accessing a data processing architecture associated with a neural network to determine dependencies between intermediate data layers of the neural network; obtaining dimensions of the intermediate data layers in the neural network; calculating a minimum number of data storage portions for executing the neural network based on the dependencies; determining a memory allocation size for each respective data storage portion of the data storage portions based on the dimensions and dependencies; allocating memory on a storage device for each data storage portion in accordance with its respective determined memory allocation size. ## Claims ### Claim 1 A method comprising: accessing a data processing architecture associated with a neural network to determine dependencies between intermediate data layers of the neural network, the intermediate data layers comprising interconnected nodes of the neural network; obtaining dimensions of the intermediate data layers in the neural network; calculating a minimum number of data storage portions for executing the neural network based on the dependencies, by: segmenting the neural network into time-based processing slices; for each time-based processing slice, identifying a number of intermediate data layers needed during execution of the neural network at the time-based processing slice; and defining the minimum number of data storage portions to equal a largest number of the identified numbers of intermediate data layers needed during execution of the neural network at the time-based processing slices, wherein each data storage portion comprises a memory location configured to store, during execution of the neural network, at least one of the intermediate data layers based on the dependencies; assigning each of the intermediate data layers to one of the minimum number of data storage portions; determining a memory allocation size for each respective data storage portion of the data storage portions based on the dimensions and dependencies; allocating memory on a storage device for each data storage portion in accordance with its respective determined memory allocation size. ### Claim 2 (depends on 1) The method of claim 1, further comprising: for each respective intermediate data layer of the intermediate data layers: designating a data storage portion of the data storage portions for the respective intermediate data layer, wherein the data storage portions are stored on a single data storage device. ### Claim 3 (depends on 2) The method of claim 2, wherein a first data storage portion of the data storage portions is designated for a first intermediate data layer and a second data storage portion is designated for a second intermediate data layer. ### Claim 4 (depends on 2) The method of claim 2, wherein a first data storage portion is designated for a plurality of intermediate data layers. ### Claim 5 (depends on 4) The method of claim 4, further comprising determining a memory allocation size for the first data storage portion based on the dimensions of the plurality of intermediate data layers. ### Claim 6 (depends on 5) The method of claim 5, wherein the memory allocation size for the first data storage portion is based on a largest total size of an intermediate data layer in the plurality of intermediate data layers that are assigned to the first data storage portion. ### Claim 7 (depends on 1) The method of claim 1, wherein allocating memory on the storage device for each memory data storage portion in accordance with its respective allocation size comprises: linearly allocating the memory on the storage device for a respective memory portion. ### Claim 8 (depends on 1) The method of claim 1, wherein allocating memory on the storage device for each memory data storage portion in accordance with its respective allocation size comprises: allocating a single contiguous block of memory on the storage device. ### Claim 9 (depends on 1) The method of claim 1, wherein accessing the data processing architecture associated with the neural network comprises: accessing metadata of the neural network that identifies the intermediate data layers and data dependencies between the intermediate data layers. ### Claim 10 (depends on 9) The method of claim 9, wherein the dimensions of the intermediate data layers are stored in the metadata. ### Claim 11 (depends on 1) The method of claim 1, wherein the determined memory allocation size for each respective data storage portion is stored as memory allocation configuration data associated with an executing architecture. ### Claim 12 (depends on 11) The method of claim 11, wherein the memory allocation configuration data is stored on a plurality of computing devices with the same executing architecture. ### Claim 13 A non-transitory computer readable medium comprising instructions, which when executed by at least one processor, configure the processor to perform operations comprising: accessing a data processing architecture associated with a neural network to determine dependencies between intermediate data layers of the neural network, the intermediate data layers comprising interconnected nodes of the neural network; obtaining dimensions of the intermediate data layers in the neural network; calculating a minimum number of data storage portions for executing the neural network based on the dependencies, by: segmenting the neural network into time-based processing slices; for each time-based processing slice, identifying a number of intermediate data layers needed during execution of the neural network at the time-based processing slice; and defining the minimum number of data storage portions to equal a largest number of the identified numbers of intermediate data layers needed during execution of the neural network at the time-based processing slices, wherein each data storage portion comprises a memory location configured to store, during execution of the neural network, at least one of the intermediate data layers based on the dependencies; assigning each of the intermediate data layers to one of the minimum number of data storage portions; determining a memory allocation size for each respective data storage portion of the data storage portions based on the dimensions and dependencies; allocating memory on a storage device for each data storage portion in accordance with its respective determined memory allocation size. ### Claim 14 (depends on 13) The non-transitory computer readable medium of claim 13, further comprising instructions, which when executed by the at least one processor, further configure the processor to perform operations comprising: for each respective intermediate data layer of the intermediate data layers: designating a data storage portion of the data storage portions for the respective intermediate data layer. ### Claim 15 (depends on 14) The non-transitory computer readable medium of claim 14, wherein a first data storage portion of the data storage portion is designated for a first intermediate data layer and a second data storage portion is designated for a second intermediate data layer. ### Claim 16 (depends on 14) The non-transitory computer readable medium of claim 14, wherein a first data storage portion is designated for a plurality of intermediate data layers. ### Claim 17 A system comprising: a storage device; at least one processor, comprising instructions, which when executed by the at least one processor, configure the processor to perform operations comprising: accessing a data processing architecture associated with a neural network to determine dependencies between intermediate data layers of the neural network; obtaining dimensions of the intermediate data layers in the neural network, the intermediate data layers comprising interconnected nodes of the neural network; calculating a minimum number of data storage portions for executing the neural network based on the dependencies, by: segmenting the neural network into time-based processing slices; for each time-based processing slice, identifying a number of intermediate data layers needed during execution of the neural network at the time-based processing slice; and defining the minimum number of data storage portions to equal a largest number of the identified numbers of intermediate data layers needed during execution of the neural network at the time-based processing slices, wherein each data storage portion comprises a memory location configured to store, during execution of the neural network, at least one of the intermediate data layers based on the dependencies; assigning each of the intermediate data layers to one of the minimum number of data storage portions; determining a memory allocation size for each respective data storage portion of the data storage portions based on the dimensions and dependencies; allocating memory on the storage device for each data storage portion in accordance with its respective determined memory allocation size. ### Claim 18 (depends on 17) The system of claim 17, the operations further comprising: for each respective intermediate data layer of the intermediate data layers: designating a data storage portion of the data storage portions for the respective intermediate data layer. ### Claim 19 (depends on 18) The system of claim 18, wherein a first data storage portion of the data storage portion is designated for a first intermediate data layer and a second data storage portion is designated for a second intermediate data layer. ### Claim 20 (depends on 18) The system of claim 18, wherein a first data storage portion is designated for a plurality of intermediate data layers.