# US 12,223,211 B2 · Enhanced input of machine-learning accelerator activations

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,223,211 B2
- Document ID: US12223211B2
- Status: Granted
- Kind: B2
- Application number: 18/468,630
- Filed: 2023-09-15
- Published: 2024-06-13
- Granted: 2025-02-11
- Earliest claimed: 2019-11-13
- Assignees: Google Inc.
- Applicants: Google LLC
- Inventors: Lukasz Lew; Wren Romano
- Web page: https://patentgrep.com/patent/US12223211B2

## CPC

- G06F 3/0679: {Non-volatile semiconductor memory device, e.g. flash memory, one time programmable memory [OTP]}
- G06F 3/0604: {Improving or facilitating administration, e.g. storage management}
- G06F 3/0655: {Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices}
- G06F 9/3001: {Arithmetic instructions}
- G06F 9/30036: {Instructions to perform operations on packed data, e.g. vector, tile or matrix operations}
- G06F 9/50: Allocation of resources, e.g. of the central processing unit [CPU]
- G06F 12/0207: {with multidimensional access, e.g. row/column, matrix}
- G06F 12/0813: with a network or matrix configuration
- G06F 12/0859: {with reload from main memory}
- G06F 12/0888: using selective caching, e.g. bypass
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/063: using electronic means
- G06N 20/00: Machine learning
- G06F 12/0862: with prefetch
- G06F 2212/1024: Latency reduction
- G06F 2212/401: Compressed data
- G06F 2212/454: Vector or matrix data
- G06F 2212/507: using speculative control
- G06F 2212/6026: Prefetching based on access pattern detection, e.g. stride based prefetch
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for scheduling operations on a machine-learning accelerator having multiple tiles. The apparatus includes a processor having a plurality of tiles and scheduling circuitry that is configured to select a respective input activation for each tile of the plurality of tiles from either an activation line for the tile or a delay register for the activation line.

## Claims

### Claim 1

A method performed using a processor comprising a plurality of activation lines and a plurality of delay registers, wherein each delay register is paired with a respective activation line, the method comprising:
on a first global tick, populating a first delay register with a first activation value from a first activation line of the processor, and
routing a second activation value to the first activation line;
on a first local tick of the first global tick, providing at least the first activation value from the delay register to a compute tile of the processor; and
on each subsequent local tick of the first global tick, providing one or more additional activation values from the plurality of activation lines after providing at least the first activation value from the delay register.

### Claim 2 (depends on 1)

The method of claim 1, wherein on each local tick of the first global tick, performing, by the processor, a mathematical operation between an input activation value and a respective weight to generate a partial sum value.

### Claim 3 (depends on 2)

The method of claim 2, wherein the mathematical operation is part of a convolution operation, and wherein the respective weights are from a convolution kernel.

### Claim 4 (depends on 2)

The method of claim 2, comprising computing, by the processor, an accumulated sum at least in part from the partial sum value and an additional partial sum value.

### Claim 5 (depends on 1)

The method of claim 1, further comprising, on a second global tick, populating one or more delay registers with one or more third activation values from the plurality of activation lines, and reading, from a memory device, one or more fourth activation values onto the plurality of activation lines.

### Claim 6 (depends on 5)

The method of claim 5, wherein the memory device comprises static random-access memory.

### Claim 7 (depends on 1)

The method of claim 1, comprising determining each activation value based on a stride length.

### Claim 8 (depends on 7)

The method of claim 7, wherein the stride length is based on a size of a convolution kernel.

### Claim 9 (depends on 1)

The method of claim 1, wherein providing one or more additional activation values from the plurality of activation lines comprises:
determining whether there are more partial sums for the first global tick; and
computing a subsequent partial sum with fewer delay registers when there are more partial sums for the first global tick, or determining whether there are more global ticks when there are no more partial sums for the first global tick.

### Claim 10 (depends on 9)

The method of claim 9, comprising overwriting the plurality of delay registers when there are more global ticks, or ending a process when there are no more global ticks.