US 11,899,746 B2 · Circuitry for high-bandwidth, low-latency machine learning

Abstract

The present disclosure relates generally to techniques for efficiently performing operations associated with artificial intelligence (AI), machine learning (ML), and/or deep learning (DL) applications, such as training and/or interference calculations, using an integrated circuit device. More specifically, the present disclosure relates to an integrated circuit design implemented to perform these operations with low latency and/or a high bandwidth of data. For example, embodiments of a computationally dense digital signal processing (DSP) circuitry, implemented to efficiently perform one or more arithmetic operations (e.g., a dot-product) on an input are disclosed. Moreover, embodiments described herein may relate to layout, design, and data scheduling of a processing element array implemented to compute matrix multiplications (e.g., systolic array multiplication).

Claims20

Dependent claims

Circuitry implemented on an integrated circuit device, comprising:

input circuitry configurable to receive a first vector of a first matrix, a second vector of the first matrix, and a third vector of a second matrix;
a register communicatively coupled to the input circuitry, wherein the register is configurable to store at least a portion of the second vector;
multiplier circuitry comprising:
a first multiplier configurable to:
receive the first vector directly from the input circuitry and receive the second vector from the register;
and
determine a first partial inner-product based at least in part on the first vector and the second vector;
and
a second multiplier configurable to determine a second partial inner-product based at least in part on the second vector and the third vector;
and
adder circuitry configurable to determine a first inner-product of a dot-product based at least in part on the first partial inner-product.

8 more elements

An integrated circuit device comprising:

a cascade chain implemented using a plurality of digital signal processing (DSP) blocks, wherein the cascade chain is configurable to determine a dot-product based at least in part on first data associated with a first matrix and second data associated with a second matrix, wherein the first data comprises a first vector and a second vector of the first matrix, wherein the second data comprises a third vector of the second matrix, wherein a first DSP block of the plurality of DSP blocks comprises:
input circuitry configurable to receive a first vector of a first matrix, a second vector of the first matrix, and a third vector of a second matrix;
a register communicatively coupled to the input circuitry, wherein the register is configurable to store at least a portion of the second vector;
multiplier circuitry comprising:
a first multiplier configurable to:
receive the first vector directly from the input circuitry and receive the second vector from the register;
and
determine a first partial inner-product based at least in part on the first vector and the second vector;
and
a second multiplier configurable to determine a second partial inner-product based at least in part on the second vector and the third vector;
and
adder circuitry configurable to determine a first inner-product of a dot-product based at least in part on the first partial inner-product;
and
an embedded memory communicatively coupled to an output of the cascade chain and configurable to:
receive the dot-product from the cascade chain;
and
store the dot-product.

14 more elements

Dependencies are inferred from the claim text.

Circuitry for high-bandwidth, low-latency machine learning

Life of this patent

Circuitry for high-bandwidth, low-latency machine learning

Life of this patent

Abstract

Claims20