# US 2021/0240797 A1 · Accelerator for dense and sparse matrix computations > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0240797 A1 - Document ID: US20210240797A1 - Status: Published application - Kind: A1 - Application number: 16/862,370 - Filed: 2020-04-29 - Published: 2021-08-05 - Earliest claimed: 2020-01-31 - Assignees: Microsoft Technology Licensing, LLC - Applicants: Microsoft Technology Licensing, LLC - Inventors: Layali RASHID; Saurabh M. KULKARNI; Marc TREMBLAY - Web page: https://patentgrep.com/patent/US20210240797A1 ## CPC - G06F 17/16: Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)} - G06F 9/3001: {Arithmetic instructions} - G06F 9/30036: {Instructions to perform operations on packed data, e.g. vector, tile or matrix operations} - G06F 9/30076: {to perform miscellaneous control operations, e.g. NOP} - G06F 9/3836: {Instruction issuing, e.g. dynamic instruction scheduling or out of order instruction execution} - G06N 3/063: using electronic means - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract A method of increasing computer hardware efficiency of a matrix computation. The method comprises receiving at a computer processing device, digital signals encoding one or more operations of the matrix computation, each operation including one or more operands. The method further comprises, responsive to determining, by a sparse data check device of the computer processing machine, that an operation of the matrix computation includes all dense operands, forwarding the operation to a dense computation device of the computer processing machine configured to perform the operation of the matrix computation based on the dense operands. The method further comprises, responsive to determining, by the sparse data check device, that an operation of the matrix computation includes one or more sparse operands, forwarding the operation to a sparse computation device configured to perform the operation of the matrix computation. ## Claims ### Claim 1 A method of increasing computer hardware efficiency of a matrix computation, comprising: receiving, at a computer processing device, digital signals encoding one or more operations of the matrix computation, each operation including one or more operands; responsive to determining, by a sparse data check device of the computer processing machine, that an operation of the matrix computation includes all dense operands, forwarding the operation to a dense computation device of the computer processing machine configured to perform the operation of the matrix computation based on the dense operands; and responsive to determining, by the sparse data check device, that the operation of the matrix computation includes one or more sparse operands, forwarding the operation to a sparse computation device configured to perform the operation of the matrix computation. ### Claim 2 (depends on 1) The method of claim 1, wherein the sparse data check device is configured to determine whether operands are sparse or dense based on determining a zero-valued operand to be sparse and a non-zero valued operand to be dense. ### Claim 3 (depends on 1) The method of claim 1, wherein the sparse data check device is configured to determine whether operands are sparse or dense based on determining an operand to be dense if a value of the operand exceeds a pre-defined threshold and sparse if the value of the operand does not exceed the pre-defined threshold. ### Claim 4 (depends on 3) The method of claim 3, wherein the pre-defined threshold is a hardware hyperparameter of the sparse data check device. ### Claim 5 (depends on 1) The method of claim 1, wherein the matrix computation is a matrix multiplication. ### Claim 6 (depends on 1) The method of claim 1, wherein the dense computation device is configured to perform a multiply-and-accumulate operation. ### Claim 7 (depends on 1) The method of claim 1, wherein the matrix computation is a neural network computation. ### Claim 8 (depends on 1) The method of claim 1, wherein the sparse computation device is configured to automatically save a sparse result value to a location derived from the operation of the matrix computation. ### Claim 9 (depends on 1) The method of claim 1, wherein the sparse computation device is configured to replace an executable instruction of the operation with a no-op instruction. ### Claim 10 (depends on 1) The method of claim 1, wherein forwarding the operation having all dense operands to the dense computation device includes enqueuing the operation having all dense operands into a lookahead dense instruction queue, wherein the dense computation device is configured to execute operations from the lookahead dense instruction queue in order. ### Claim 11 (depends on 10) The method of claim 10, further comprising feeding the lookahead dense instruction queue in excess of a number of operations the dense computation device is configured to process in a subsequent cycle. ### Claim 12 (depends on 1) The method of claim 1, wherein forwarding the operation having one or more sparse operands to the sparse computation device includes enqueuing the operation having one or more sparse operands into a lookahead sparse instruction queue. ### Claim 13 (depends on 12) The method of claim 12, wherein the sparse computation device is configured to automatically store sparse result values from operations in the lookahead sparse instruction queue, in a program order of the operations in the lookahead sparse instruction queue. ### Claim 14 (depends on 12) The method of claim 12, further comprising feeding the lookahead sparse instruction queue in excess of a number of operations the sparse computation device is configured to process in a subsequent cycle. ### Claim 15 A computer system for performing matrix computations, including: a sparse computation device configured to calculate a result of an operation having one or more sparse operands; a dense computation device configured to calculate a result of an operation having all dense operands; an instruction issue stage configured to receive digital signals encoding one or more operations of a matrix computation; and a sparse data check device configured to forward an operation having one or more sparse operands to the sparse computation device, and to forward an operation having all dense operands to the dense computation device. ### Claim 16 (depends on 15) The computer system of claim 15, wherein the sparse data check device is configured to determine an operand is dense if a value of the operand is greater than the pre-defined threshold, and configured to determine the operand is sparse if the value of the operand is less than the pre-defined threshold. ### Claim 17 A computer system for performing matrix computations, including: a sparse computation unit configured to perform a matrix computation on sparse data; a dense computation unit configured to perform the matrix computation on dense data; an instruction issue stage configured to receive a plurality of instructions including an instruction operating on sparse data and an instruction operating on dense data; and a sparse data check device configured to distinguish between instructions operating on sparse data and instructions operating on dense data, wherein the sparse data check device is further configured to forward instructions determined to operate on sparse data to the sparse computation device and to forward instructions determined to operate on dense data to the dense computation device. ### Claim 18 (depends on 17) The computer system of claim 17, wherein the sparse data check device is configured to detect a plurality of instructions operating on sparse data and enqueue the plurality of instructions operating on sparse data into a lookahead sparse instruction queue. ### Claim 19 (depends on 17) The computer system of claim 17, wherein the sparse data check device is configured to detect a plurality of instructions operating on dense data and enqueue the plurality of instructions operating on dense data into a lookahead dense instruction queue. ### Claim 20 (depends on 17) The computer system of claim 17, wherein the sparse computation device is configured to forward instruction identifiers to a sparse register identifier queue, and the sparse register identifier queue is configured to write corresponding results according to a program-order associated with the instruction identifiers.