Back to results
US 2025/0117873 A1Published application
Machine learning sparse computation mechanism
AssigneeIntel Corporation
InventorsEriko Nurvitadhi, Balaji Vembu, Tsung-Han Lin, Kamal Sinha, Rajkishore Barik, Nicolas C. Galoppo Von Borries
Life of this patent
Estimating the term…
Abstract
Techniques to improve performance of matrix multiply operations are described in which a compute kernel can specify one or more element-wise operations to perform on output of the compute kernel before the output is transferred to higher levels of a processor memory hierarchy.
Claims21
An apparatus comprising:
- a system interconnect;
- an accelerator device coupled with the system interconnect, the accelerator device including:
- first circuitry to load elements of matrices into a first memory of the accelerator device, wherein the first memory is a global memory;
- second circuitry to transfer a subset of elements of the matrices from the first memory into a second memory, the second memory local to a set of processing resources of the accelerator device;
- third circuitry configurable to execute a compute kernel on the accelerator device, the compute kernel to perform a matrix multiply operation on the subset of elements of the matrices, the compute kernel to specify one or more element-wise operations to perform on output of the matrix multiply operation before the output is transferred to the first memory, and the one or more element-wise operations include to apply an activation function to the output;
- and
- fourth circuitry to perform the one or more element-wise operations on the output of the matrix multiply operation while the output is stored in the second memory.
4 more elements
A non-transitory machine-readable medium including instructions, execution of which by one or more processors, causes the one or more processors to perform operations comprising:
- loading elements of matrices into a first memory of an accelerator device of the one or more processors, wherein the first memory is a global memory of the accelerator device;
- transferring a subset of elements of the matrices from the first memory into a second memory of the accelerator device, the second memory local to a set of processing resources of the accelerator device;
- and
- triggering execution of a compute kernel on the accelerator device, wherein the compute kernel performs a matrix multiply operation on the subset of elements of the matrices, and a matrix multiply framework enables the compute kernel to specify one or more element-wise operations to perform on output of the matrix multiply operation before the output is transferred to the first memory, the one or more element-wise operations including applying an activation function to the output and the matrix multiply framework is to provide a primitive to enable the compute kernel to specify the one or more element-wise operations to perform on the output of the matrix multiply operation.
1 more element
11-12. (canceled)
A data processing system comprising:
- a memory device;
- an accelerator device coupled with the memory device, the accelerator device
- including:
- first circuitry to load elements of matrices into a first memory of the accelerator device, wherein the first memory is a global memory;
- second circuitry to transfer a subset of elements of the matrices from the first memory into a second memory, the second memory local to a set of processing resources of the accelerator device;
- third circuitry configurable to execute a compute kernel on the accelerator device, the compute kernel to perform a matrix multiply operation on the subset of elements of the matrices, the compute kernel to specify one or more element-wise operations to perform on output of the matrix multiply operation before the output is transferred to the first memory, and the one or more element-wise operations include to apply an activation function to the output;
- and
- fourth circuitry to perform the one or more element-wise operations on the output of the matrix multiply operation while the output is stored in the second memory.
5 more elements
Dependencies are inferred from the claim text.