# US 2025/0094220 A1 · Processor instruction dispatch configuration

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0094220 A1
- Document ID: US20250094220A1
- Status: Published application
- Kind: A1
- Application number: 18/968,692
- Filed: 2024-12-04
- Published: 2025-03-20
- Earliest claimed: 2019-11-18
- Assignees: Groq, Inc.
- Applicants: Groq, Inc.
- Inventors: Brian Lee Kurtz; Dinesh Maheshwari; James David Sproch
- Web page: https://patentgrep.com/patent/US20250094220A1

## CPC

- G06F 5/01: for shifting, e.g. justifying, scaling, normalising {(digital stores in which the information is moved stepwise, e.g. shift-registers G11C19/00; digital stores in which the information circulates G11C21/00)}
- G06F 9/4881: {Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues}
- G06F 9/3001: {Arithmetic instructions}
- G06F 9/30032: {Movement instructions, e.g. MOVE, SHIFT, ROTATE, SHUFFLE}
- G06F 9/30036: {Instructions to perform operations on packed data, e.g. vector, tile or matrix operations}
- G06F 9/3802: {Instruction prefetching}
- G06F 9/3836: {Instruction issuing, e.g. dynamic instruction scheduling or out of order instruction execution}
- G06F 9/3856: {Reordering of instructions, e.g. using queues or age tags}
- G06F 9/3885: {using a plurality of independent parallel functional units}
- G06F 9/3887: {controlled by a single instruction for multiple data lanes [SIMD]}
- G06F 15/8046: {Systolic arrays}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A processor comprises a computational array of computational elements and an instruction dispatch circuit. The computational elements receive data operands via data lanes extending along a first dimension, and processes the operands based upon instructions received from the instruction dispatch circuit via instruction lanes extending along a second dimension. The instruction dispatch circuit receives raw instructions, and comprises an instruction dispatch unit (IDU) processor that processes a set of raw instructions to generate processed instructions for dispatch to the computational elements, where the number of processed instructions is not equal to the number of instructions of the set of raw instructions. The processed instructions are dispatched to columns of the computational array via a plurality of instruction queues, wherein an instruction vector of instructions is shifted between adjacent instruction queues in a first direction, and dispatches instructions to the computational elements in a second direction.

## Claims

### Claim 1

A system, comprising:
at least one processor, comprising:
an array of computational elements;
a memory that stores data operands and instruction data to be processed by the array of computational elements, and, based on a defined temporal relationship, provides the data operands to respective computational elements of the array of computational elements; and
an instruction dispatch circuit that, based on shifting data of the instruction data along a first direction and a second direction, provides the instruction data to the array of computational elements, wherein the first direction and the second direction are relative to a direction of flow of data in the at least one processor.

### Claim 2 (depends on 1)

The system of claim 1, wherein the at least one processor further comprises:
a compiler that, based on the defined temporal relationship, generates a compiled program that indicates a timing at which the data operands and the instruction data are read from the memory and provided to the array of computational elements.

### Claim 3 (depends on 1)

The system of claim 1, wherein the at least one processor further comprises:
a control circuit that controls operations of the memory and the instruction dispatch circuit.

### Claim 4 (depends on 1)

The system of claim 1, wherein the respective computational elements of the array of computational elements are configured to process the data operands based upon the instruction data.

### Claim 5 (depends on 1)

The system of claim 1, wherein the instruction dispatch circuit shifts the data of the instruction data along the first direction and the second direction in a staggered manner.

### Claim 6 (depends on 1)

The system of claim 1, wherein the first direction is parallel to a direction of a flow of data in the at least one processor, and wherein the second direction is perpendicular to the direction of the flow of data in the at least one processor.

### Claim 7 (depends on 1)

The system of claim 1, wherein, during defined timing increments of the defined temporal relationship, the instruction data are moved only in the first direction.

### Claim 8 (depends on 1)

The system of claim 1, wherein, during defined timing increments of the defined temporal relationship, the instruction data are moved only in the second direction.

### Claim 9 (depends on 1)

The system of claim 1, wherein the data operands correspond to weights utilized to implement a model.

### Claim 10 (depends on 1)

The system of claim 1, wherein the data operands correspond to activations utilized to implement a model.

### Claim 11 (depends on 1)

The system of claim 1, wherein the defined temporal relationship is during a same cycle.

### Claim 12 (depends on 1)

The system of claim 1, wherein the defined temporal relationship is separated by a defined delay.

### Claim 13

A method, comprising:
storing, in a memory, data operands and instruction data to be processed by an array of computational elements of a processor;
based on a defined temporal relationship, providing data operands to respective computational elements of the array of computational elements; and
shifting data of the instruction data along a first direction and a second direction, wherein the shifting comprises providing the instruction data to the array of computational elements, and wherein the first direction and the second direction are relative to a direction of flow of data in the processor.

### Claim 14 (depends on 13)

The method of claim 13, further comprising:
prior to the providing the data operands, and based on the defined temporal relationship, generating a compiled program that indicates a timing at which the data operands and the instruction data are read from the memory and provided to the array of computational elements.

### Claim 15 (depends on 14)

The method of claim 14, wherein the generating comprises determining the timing based on a hardware configuration of the processor.

### Claim 16 (depends on 14)

The method of claim 14, wherein the generating comprises determining the timing based on a type of instruction included in the instruction data.

### Claim 17 (depends on 13)

The method of claim 13, wherein the shifting the data comprises shifting the data of the instruction data along the first direction and the second direction in a staggered manner.

### Claim 18 (depends on 13)

The method of claim 13, wherein the first direction is parallel to a direction of a flow of data in the processor, and wherein the second direction is perpendicular to the direction of the flow of data in the processor.

### Claim 19 (depends on 13)

The method of claim 13, further comprising:
during defined timing increments of the defined temporal relationship, moving the instruction data only in the first direction.

### Claim 20 (depends on 13)

The method of claim 13, further comprising:
during defined timing increments of the defined temporal relationship, moving the instruction data only in the second direction.
