# US 12,007,937 B1 · Multi-mode architecture for unifying matrix multiplication, 1×1 convolution and 3×3 convolution > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,007,937 B1 - Document ID: US12007937B1 - Status: Granted - Kind: B1 - Application number: 18/523,632 - Filed: 2023-11-29 - Granted: 2024-06-11 - Earliest claimed: 2023-11-29 - Assignees: Recogni Inc. - Applicants: Recogni Inc. - Inventors: Jian hui Huang; Gary S. Goldman - Web page: https://patentgrep.com/patent/US12007937B1 ## CPC - G06F 7/5443: {Sum of products (for applications thereof, see the relevant places, e.g. G06F17/10, H03H17/00)} - G06F 15/80: comprising an array of processing units with common control, e.g. single instruction multiple data processors (G06F15/82 takes precedence {; for correlation function computation G06F17/15}) - G06F 9/544: {Buffers; Shared memory; Pipes} - G06F 15/8046: {Systolic arrays} - G06F 17/16: Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract In a system with control logic and a processing element array, two modes of operation may be provided. In the first mode of operation, the control logic may configure the system to perform matrix multiplication or 1×1 convolution. In the second mode of operation, the control logic may configure the system to perform 3×3 convolution. The processing element array may include an array of processing elements. Each of the processing elements may be configured to compute the dot product of two vectors in a single clock cycle, and further may accumulate the dot products that are sequentially computed over time. ## Claims ### Claim 1 A system, comprising: first and second memories; first, second and third processing elements; means for communicating data from the first memory to the first, second and third processing elements; and means for communicating data from the second memory to the first, second and third processing elements in one of a first or second configuration, wherein: in the first configuration, first, second and third groups of values from the second memory are transmitted to the first, second and third processing elements, respectively, and three values are shared between the first, second and third groups of values from the second memory, in the second configuration, fourth, fifth and sixth groups of values from the second memory are transmitted to the first, second and third processing elements, respectively, the first processing element is configured to manipulate the data from the first memory with the first or fourth groups of values from the second memory so as to compute a first dot product, the second processing element is configured to manipulate the data from the first memory with the second or fifth groups of values from the second memory so as to compute a second dot product, and the third processing element is configured to manipulate the data from the first memory with the third or sixth groups of values from the second memory so as to compute a third dot product. ### Claim 2 (depends on 1) The system of claim 1, wherein the means for communicating data from the first memory to the first, second and third processing elements comprises a first buffer that is communicatively coupled to the first memory and the first, second and third processing elements, and wherein the means for communicating data from the second memory to the first, second and third processing elements in the first or second configuration comprises a second buffer that is communicatively coupled to the second memory and the first, second and third processing elements. ### Claim 3 (depends on 2) The system of claim 2, wherein the second buffer comprises one or more shift registers for storing the first, second and third groups of values from the second memory. ### Claim 4 (depends on 2) The system of claim 2, wherein the second buffer comprises a double buffer for storing the fourth, fifth and sixth groups of values from the second memory. ### Claim 5 (depends on 2) The system of claim 2, wherein the means for communicating data from the second memory to the first, second and third processing elements in the first or the second configuration further comprises a plurality of multiplexors configured to selectively transmit respective outputs of the second buffer to respective inputs of the first, second and third processing elements. ### Claim 6 (depends on 2) The system of claim 2, wherein the means for communicating data from the second memory to the first, second and third processing elements in the first or the second configuration further comprises wires which directly couple respective outputs of the second buffer to respective inputs of the first, second and third processing elements. ### Claim 7 (depends on 1) The system of claim 1, wherein the first, second and third groups of values from the second memory each consists of nine values. ### Claim 8 (depends on 1) The system of claim 1, wherein each of the first, second and third processing elements includes a plurality of two-input multipliers, and an adder tree configured to sum respective outputs of the plurality of two-input multipliers. ### Claim 9 (depends on 8) The system of claim 8, wherein each of the first, second and third processing elements includes a storage element configured to store an output of the adder tree. ### Claim 10 (depends on 9) The system of claim 9, wherein for each of the first, second and third processing elements, the adder tree is further configured to receive an output of the storage element. ### Claim 11 A method, comprising: communicating data from a first memory to first, second and third processing elements; selecting, by a controller, one of a first or second configuration of an integrated circuit for transmitting data from a second memory to the first, second and third processing elements, the integrated circuit being controllable to operate in the first configuration and being controllable to operate in the second configuration, wherein: in the first configuration, first, second and third groups of values from the second memory are transmitted to the first, second and third processing elements, respectively, and three values are shared between the first, second and third groups of values from the second memory, and in the second configuration, fourth, fifth and sixth groups of values from the second memory are transmitted to the first, second and third processing elements, respectively; communicating data from the second memory to the first, second and third processing elements in the selected configuration of the integrated circuit; manipulating, by the first processing element, the data from the first memory with the first or fourth groups of values from the second memory so as to compute a first dot product; manipulating, by the second processing element, the data from the first memory with the second or fifth groups of values from the second memory so as to compute a second dot product; and manipulating, by the third processing element, the data from the first memory with the third or sixth groups of values from the second memory so as to compute a third dot product. ### Claim 12 (depends on 11) The method of claim 11, wherein the data from the first memory is communicated to the first, second and third processing elements via a first buffer, and wherein the data from the second memory is communicated to the first, second and third processing elements via a second buffer. ### Claim 13 (depends on 12) The method of claim 12, further comprising storing the first, second and third groups of values from the second memory in a shift register of the second buffer. ### Claim 14 (depends on 12) The method of claim 12, further comprising storing the fourth, fifth and sixth groups of values from the second memory in a double buffer of the second buffer. ### Claim 15 (depends on 12) The method of claim 12, wherein a portion of the data from the second memory is communicated to the first, second and third processing elements via a plurality of multiplexors configurably coupling respective outputs of the second buffer to respective inputs of the first, second and third processing elements. ### Claim 16 (depends on 12) The method of claim 12, wherein a portion of the data from the second memory is communicated to the first, second and third processing elements via wires directly coupling respective outputs of the second buffer with respective inputs of the first, second and third processing elements. ### Claim 17 (depends on 11) The method of claim 11, wherein the first, second and third groups of values from the second memory each consists of nine values. ### Claim 18 (depends on 11) The method of claim 11, wherein each of the first, second and third processing elements includes a plurality of two-input multipliers, and an adder tree configured to sum respective outputs of the plurality of two-input multipliers. ### Claim 19 (depends on 18) The method of claim 18, wherein each of the first, second and third processing elements includes a storage element configured to store an output of the adder tree. ### Claim 20 (depends on 19) The method of claim 19, wherein for each of the first, second and third processing elements, the adder tree is further configured to receive an output of the storage element.