US 10,896,045 B2Granted

Architecture for dense operations in machine learning inference engine

AssigneeMarvell Asia Pte Ltd

InventorsAvinash Sodani, Ulf Hanebutte, Senad Durakovic, Hamid Reza Ghasemi, Chia-Hsin Chen

Life of this patent

Estimating the term…

Loading patent…

Abstract

A processing unit of an inference engine for machine learning (ML) includes a first, a second, and a third register, and a matrix multiplication block. The first register receives a first stream of data associated with a first matrix data that is read only once. The second register receives a second stream of data associated with a second matrix data that is read only once. The matrix multiplication block performs a multiplication operation based on data from the first register and the second register resulting in an output matrix. A row associated with the first matrix is maintained while rows associated with the second matrix is fed to the matrix multiplication block to perform a multiplication operation. The process is repeated for each row of the first matrix. The third register receives the output matrix from the matrix multiplication block and stores the output matrix.

Claims25

Dependent claims

A processing unit of an inference engine for machine learning (ML), comprising:

a first bank of registers configured to receive a first stream of data associated with a first matrix data, and wherein the first matrix data is read into the first bank of registers only once;
a second bank of registers configured to receive a second stream of data associated with a second matrix data, and wherein the second matrix data is read into the second bank of registers only once;
a matrix multiplication block configured to perform a multiplication operation based on data received from the first bank of registers and the second bank of registers and output results of the multiplication operation as an output matrix,
wherein each register of the first bank of registers associated with a row of the first matrix stays the same while columns of the second bank of registers associated with the second matrix are fed into the matrix multiplication block one column at a time and multiplied by the each register the first bank of registers associated with the row of the first matrix to generate a partial result of the multiplication operation, wherein each register of the second bank of registers is associated with a column of the second matrix;
wherein, at the same time, each register of the second bank of registers associated with a column of the second matrix stays the same while rows of the first bank of registers associated with the first matrix are fed into the matrix multiplication block one row at a time to be multiplied by the each register of the second bank of registers associated with the column of the second matrix to generate another partial result of the multiplication operation;
and
a third bank of registers configured to receive and accumulate the partial results of the multiplication operation from the matrix multiplication block until the multiplication operation is complete.

4 more elements

A processing unit of an inference engine for machine learning (ML), comprising:

a first register configured to receive a first stream of data associated with a first matrix data, wherein the first matrix data is read only once;
a second register configured to receive a second stream of data associated with a second matrix data, wherein the second matrix data is read only once;
a matrix multiplication block configured to perform a multiplication operation based on data from the first register and the second register, wherein the multiplication operation results in an output matrix, and wherein a row associated with the first matrix is maintained while columns associated with the second matrix are fed to the matrix multiplication block to perform a multiplication operation and wherein the maintaining of rows of the first matrix and the feeding of the columns of the second matrix are repeated for each row of the first matrix until a partial output matrix is generated, and wherein a column associated with the second matrix is maintained while rows associated with the first matrix are fed to the matrix multiplication block to perform a multiplication operation and wherein the maintaining of columns of the second matrix and the feeding of the rows of the first matrix are repeated for each column of the second matrix until another partial output matrix is generated;
and
a third register configured to receive and accumulate the partial output matrices from the matrix multiplication block until the multiplication operation is complete.

2 more elements

Dependencies are inferred from the claim text.

Facts

Application: 16/226,550Filed 19 Dec 2018 · us provisional application
Priority: 8 Feb 2018
Published: US 2019/0244130 A18 Aug 2019
Granted: 19 Jan 2021
Term ends: Not estimatedThe source records don’t support an estimate.

Classification17 codes

G06F 9/3879
Using a secondary processor, e.g. coprocessor: for non-native instruction execution, e.g. executing a command; for Java instruction set
G06F 9/30174
Runtime instruction translation, e.g. macros: for non-native instruction set, e.g. Javabyte, legacy code
G06F 9/3836
Concurrent instruction execution, e.g. pipeline or look ahead: instruction issuing, e.g. dynamic instruction scheduling or out of order…
G06F 9/3851
Instruction issuing, e.g. dynamic instruction scheduling or out of order instruction execution: from multiple instruction streams, e.g. multistreaming

Family

Application62/628,130Us provisional application · filed 8 Feb 2018
Application62/644,352Us provisional application · filed 16 Mar 2018
Application62/675,076Us provisional application · filed 22 May 2018
US 2019/0244130 A1Pre-grant publication · published 8 Aug 2019
US 10,896,045 B2This patent

From the US filing history. Foreign family members aren’t listed.