US 2019/0244118 A1 · ARCHITECTURE OF CROSSBAR OF INFERENCE ENGINE

Abstract

A programmable hardware system for machine learning (ML) includes a core and an inference engine. The core receives commands from a host. The commands are in a first instruction set architecture (ISA) format. The core divides the commands into a first set for performance-critical operations, in the first ISA format, and a second set of performance non-critical operations, in the first ISA format. The core executes the second set to perform the performance non-critical operations of the ML operations and streams the first set to inference engine. The inference engine generates a stream of the first set of commands in a second ISA format based on the first set of commands in the first ISA format. The first set of commands in the second ISA format programs components within the inference engine to execute the ML operations to infer data.

Claims20

Dependent claims

A crossbar-based inference engine configured to perform a machine learning (ML) operation on an input data stream, comprising:

a plurality of on-chip memories (OCMs) coupled to a crossbar and each OCM is configured to
load and maintain data from the input data stream for local access by components in the inference engine;
maintain and output result of the ML operation performed by the components in the inference engine as an output data stream;
a plurality of first processing units (PODs) each coupled to one OCM of the plurality of OCMs and configured to perform a dense and/or regular computation task of the ML operation on the data within the corresponding OCM;
a plurality of second processing units/elements (PEs) coupled to the crossbar and wherein each PE is configured to perform a sparse and/or irregular computation task of the ML operation on the data within the OCMs and/or from the POD units;
and
said crossbar configured to connect the plurality of PEs to the plurality of OCMs to enable each PE of the plurality of PEs to read data from and/or write data to the corresponding OCM.

4 more elements

Dependencies are inferred from the claim text.

Architecture of crossbar of inference engine

Life of this patent

Architecture of crossbar of inference engine

Life of this patent

Abstract

Claims20