US 2017/0323224 A1 · APPARATUS FOR HARDWARE ACCELERATED MACHINE LEARNING

Abstract

An architecture and associated techniques of an apparatus for hardware accelerated machine learning are disclosed. The architecture features multiple memory banks storing tensor data. The tensor data may be concurrently fetched by a number of execution units working in parallel. Each operational unit supports an instruction set specific to certain primitive operations for machine learning. An instruction decoder is employed to decode a machine learning instruction and reveal one or more of the primitive operations to be performed by the execution units, as well as the memory addresses of the operands of the primitive operations as stored in the memory banks. The primitive operations, upon performed or executed by the execution units, may generate some output that can be saved into the memory banks. The fetching of the operands and the saving of the output may involve permutation and duplication of the data elements involved.

Claims20

Dependent claims

An execution unit of hardware acceleration for machine learning, the execution unit running on a clock and configured to execute one instruction of an instruction set per clock cycle of the clock, the execution unit comprising:

a clock input receiving the clock;
an operation specification input receiving an opcode specifying the one instruction;
a first operand input receiving a first operand;
a second operand input receiving a second operand;
an accumulation register storing a current accumulation value;
and
an output,
wherein:
the accumulation register is updated, within the clock cycle, with a new accumulation value based on the opcode and one or more of the first operand, the second operand and the current accumulation value, and
the output comprises saturated lower bits of the new value.

7 more elements

An indexing module for looping over a plurality of indexes each corresponding to one of multiple dimensions of a tensor, the indexing module comprising:

a counter unit comprising:
a plurality of counter registers each storing a respective one of the plurality of indexes;
and
a plurality of size registers each corresponding to one of the counter registers;
and
an address unit,
wherein:
the counter unit updates the plurality of indexes stored in the counter registers by repeating a method to loop over the plurality of indexes, and
the address unit generates, for each of the indexes stored in the counter registers, an address that is a polynomial of the respective index.

6 more elements

A permutation network for transferring data elements between a vector of logical addresses and multiple banks of random access memory (RAM), the permutation network comprising:

a first wiring pattern configured to map a logical space to a cyclic space;
a first barrel shifter;
a second wiring pattern configured to map the cyclic space to the logical space;
and
a second barrel shifter,
wherein:
a quantity number of the data elements is less than or equal to a quantity number of the banks of RAM,
the vector of logical addresses is represented by an address offset and a multiplier,
the multiplier is relatively prime to the quantity number of the banks of RAM,
the first barrel shifter rotates the data elements in the cyclic space based on the multiplier, and
the second barrel shifter rotates the data elements in the logical space based on the address offset.

8 more elements

A banked asynchronous memory read unit for reading data from a computer readable memory having a plurality of memory banks, comprising:

a modulo address generator configured to generate:
a set of per-bank read addresses each addressing respective data to be read from a respective memory bank of the plurality of memory banks;
and
a set of per-bank enable bits indicating the one or more respective memory banks of the plurality of memory banks;
a plurality of memory queues each corresponding to a respective memory bank of the plurality of memory banks, each of the memory queues comprising:
an address queue (AQ) configured to receive the read address for the respective memory bank;
a memory reader (D) configured to read the data addressed by the read address;
and
an output data queue (DQ) configured to store the data read by the memory reader;
a control queue (CQ) configured to store the set of per-bank enable bits;
and
a deque logic controller configured to combine the data stored in the data output queues of the plurality of memory queues into complete output data based on the set of per-bank enable bits stored in the control queue.

9 more elements

Dependencies are inferred from the claim text.

Apparatus for hardware accelerated machine learning

Life of this patent

Apparatus for hardware accelerated machine learning

Life of this patent

Abstract

Claims20