Loading patent…
AssigneeNeuralmagic Inc.
InventorRati GELASHVILI
Estimating the term…
A computer processor may include a number of cores, a shared cache shared among the cores, and a local cache associated with each core and used by that core only. Input data for a neural network (NN) layer may be partitioned into a set of tiles of size T×T, and the tile set may be partitioned into blocks of R tiles. For each block, a core may perform a transform operation on the tiles to produce transformed data matrices fitting in a local cache, and a set of multiply operations, each multiply operation using a transformed data matrix and a transformed kernel matrix from a set of transformed kernel matrices. The set of transformed kernel matrices may fit in the shared cache. The result of at least one of the multiply operations may be stored in a location used to store a transformed data matrix.
A method of executing matrix multiply operations for a neural network (NN), the method comprising, using a computer processor comprising a plurality of cores and a shared cache shared among the cores, each core associated with a local cache used by that core only:
A system for of executing matrix multiply operations for a neural network (NN), the system comprising a computer processor comprising:
A method of performing inference for a neural network (NN), the method comprising, in a computer processor comprising a plurality of cores, a shared cache, and a private cache used by each core, the method comprising:
Dependencies are inferred from the claim text.