# US 2023/0055528 A1 · Systems and methods for executing a programmable finite state machine that accelerates fetchless computations and operations of an array of processing cores of an integrated circuit

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0055528 A1
- Document ID: US20230055528A1
- Status: Published application
- Kind: A1
- Application number: 17/953,312
- Filed: 2022-09-26
- Published: 2023-02-23
- Earliest claimed: 2021-08-22
- Assignees: quadric.io, Inc.
- Applicants: quadric.io Inc.
- Inventors: Thomas Ng; Nigel Drego; Daniel Firu; Veerbhan Kheterpal; Aman Sikka
- Web page: https://patentgrep.com/patent/US20230055528A1

## CPC

- G06F 30/343: Logical level
- G06F 8/443: {Optimisation}
- G06F 9/30032: {Movement instructions, e.g. MOVE, SHIFT, ROTATE, SHUFFLE}
- G06F 9/30065: {Loop control instructions; iterative instructions, e.g. LOOP, REPEAT}
- G06F 9/30141: {Implementation provisions of register files, e.g. ports}
- G06F 9/4498: {Finite state machines}
- G06F 30/27: using machine learning, e.g. artificial intelligence, neural networks, support vector machines [SVM] or training a model
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/063: using electronic means
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Systems and methods for fetchless acceleration of convolutional loops on an integrated circuit include identifying, by a compiler, finite state machine (FSM) initialization parameters based on computational requirements of a computational loop; initializing a programmable FSM based on the FSM initialization parameters, wherein the FSM initialization parameters include a loop iteration parameter identifying a number of computation cycles of the computational loop; executing the programmable FSM to enable fetchless computations by: generating a plurality of computational loop control signals including a distinct computation loop control signal for each of the number of computation cycles of the computational loop based on the loop iteration parameter; and controlling an execution of a plurality of computation cycles of a computational circuit performing the computational loop based on transmitting the plurality of computational loop control signals until the number of computation cycles of the computation loop are completed.

## Claims

### Claim 1

A method for implementing fetchless acceleration of computational loops on an integrated circuit, the method comprising:
identifying, by a compiler, finite state machine (FSM) initialization parameters based on computational requirements of a computational loop within a neural network graph;
initializing a programmable FSM based on the FSM initialization parameters, wherein the FSM initialization parameters include a loop iteration parameter comprising a number of computation cycles of the computational loop;
at runtime, executing the programmable FSM to enable fetchless computations by:
(i) generating, by the programmable FSM, a plurality of computational loop control signals including a distinct computation loop control signal for each of the number of computation cycles of the computational loop based on the loop iteration parameter; and
(ii) controlling, by the programmable FSM, an execution of a plurality of computation cycles of a computational circuit performing the computational loop based on transmitting the plurality of computational loop control signals until the number of computation cycles of the computation loop are completed.

### Claim 2 (depends on 1)

The method according to claim 1, wherein
the FSM initialization parameters further includes
a loop iteration and data movement parameter comprising (a) a distinct number of computation cycles of the computation loop and (b) at least one data movement instruction that, when executed, moves input data from a register file of a processing core to the data input ports of neighboring processing cores.

### Claim 3 (depends on 2)

The method according to claim 2, wherein
the register file is associated with one or more output data ports of a processing core and data input ports from neighboring processing cores, which are directly connected to those neighboring processing cores' data output ports; and
when the data movement instruction is executed causes the input data to rotate an angle from the input data ports to the output data ports.

### Claim 4 (depends on 1)

The method according to claim 1, wherein
implementing the programmable FSM includes:
generating, by the programmable FSM, a data movement control signal for each distinct number of computation cycles of the computation loop based on the loop iteration and data movement parameter.

### Claim 5 (depends on 4)

The method according to claim 4, wherein
controlling the execution of the plurality of computation cycles of the computational circuit includes transmitting, by the programmable FSM, the data movement control signal for each distinct number of computation cycles until the distinct number of computation cycles of the computation loop are completed.

### Claim 6 (depends on 1)

The method according to claim 1, wherein
initializing the programmable FSM based on the FSM initialization parameters includes encoding a starting memory address parameter to a start memory address register file accessible to one or more computational circuits controllable by the programmable FSM.

### Claim 7 (depends on 6)

The method according to claim 6, wherein
the starting memory address parameter comprises a register file pointer that points to a head of the input data at a location within an n-dimensional memory stored within at least one processing core controllable by the programmable FSM.

### Claim 8 (depends on 1)

The method according to claim 1, wherein
initializing the programmable FSM based on the FSM initialization parameters includes encoding a convolution filter size parameter to a convolution register file of at least one processing core controllable by the programmable FSM.

### Claim 9 (depends on 8)

The method according to claim 8, wherein
the convolution filter size parameter comprises a value that maps to one of a plurality of distinct convolutional filter sizes for a given convolutional computation by a multiply accumulator circuit of the at least one processing core.

### Claim 10 (depends on 1)

The method according to claim 1, wherein
initializing the programmable FSM based on the FSM initialization parameters includes encoding the loop iteration parameter to a combination of distinct iteration register files of at least one processing core controllable by the programmable FSM.

### Claim 11 (depends on 2)

The method according to claim 2, wherein
at runtime, the programmable FSM executes the computational loop based on the loop iteration parameter, and
subsequently, the programmable FSM executes one or more computational loops based on the loop iteration and data movement parameter.

### Claim 12 (depends on 2)

The method according to claim 2, wherein
at runtime, the programmable FSM produces:
a first set of control signals of the plurality of computational loop control signals for executing the computational loop based on the loop iteration parameter; and
in response to completing the computational loop based on the loop iteration parameter, a second set of control signals of the plurality of computational loop control signals for executing (a) the distinct number of computation cycles of the computation loop and (b) the at least one data movement instruction based on the loop iteration and data movement parameter.

### Claim 13 (depends on 1)

The method according to claim 1, wherein:
at runtime, the programmable FSM produces the plurality of controls signals causing an execution of an N-way multiply accumulate with computation weights and computation input data,
N relates to a number of distinct multiply accumulate circuits concurrently executing a distinct computational loop, and
N is greater than one.

### Claim 14 (depends on 1)

The method according to claim 1, wherein
if convolutional filter size parameter of the FSM initialization parameters includes a value that maps to one of a plurality of distinct convolutional filter sizes that is greater than a 1×1 convolutional filter size, the programmable FSM broadcasts input data pointed to by the starting memory address parameter to a collection of processing cores in neighboring proximity.

### Claim 15

A method comprising:
identifying, by a compiler, finite state machine (FSM) initialization parameters based on computational requirements of a convolutional loop within a neural network graph;
initializing a programmable FSM based on the FSM initialization parameters, wherein the FSM initialization parameters include a loop iteration parameter comprising a number of multiply-accumulate computation cycles of the convolutional loop;
at runtime, implementing the programmable FSM to enable fetchless computations by:
(i) generating, by the programmable FSM, a plurality of convolutional loop control signals based on the loop iteration parameter; and
(ii) controlling, by the programmable FSM, an execution of a plurality of multiply-accumulate computation cycles of a multiply accumulator circuit (MAC) performing the convolutional loop based on transmitting the plurality of convolutional loop control signals until the number of multiply-accumulate computation cycles of the computation loop are completed.

### Claim 16 (depends on 15)

The method according to claim 15, wherein
initializing the programmable FSM based on the FSM initialization parameters includes:
(i) programming a starting memory address parameter at a start memory address register file accessible to the MAC controllable by the programmable FSM;
(ii) programming a convolution filter size parameter at a convolution register file accessible to the MAC controllable by the programmable FSM; and
(iii) programming iteration parameters at one or more iteration registers file accessible to the programmable FSM.

### Claim 17 (depends on 15)

The method according to claim 15, wherein
the programmable FSM is in direct command signal communication with a plurality of distinct MACS operating on each of a plurality of distinct processing cores.

### Claim 18 (depends on 17)

The method according to claim 17, further comprising:
identifying, by the compiler, FSM initialization parameters includes:
computing a memory start address parameter including a memory address location within a local memory of each of the plurality of distinct processing cores.

### Claim 19

A method for implementing FSM-controlled convolutional computations on an integrated circuit, the method comprising:
identifying FSM programming instructions based on a neural network graph;
configuring a programmable FSM based on the FSM programming instructions, wherein the programmable FSM controls:
(a) operations of multiply accumulators (MACs) of a plurality of distinct processing cores, and
(b) data movement operations of data ports of the plurality of distinct processing cores; and
wherein configuring the programmable FSM includes:
(1) programming a starting memory address value to an address register file accessible to MACs controllable by the programmable FSM,
(2) programming a convolutional filter size to a convolutional register file associated with the FSM, and
(3) programming at least one iteration register file associated with the programmable FSM with an iteration value identifying a number of cycles of a convolutional loop performed by at least one of the MACs; and
executing a Boolean switch based on the initialization of the programmable FSM that starts an operation of the programmable FSM for generating control signals to the MACs for automatically executing distinct convolutional loops.
