US 12,112,174 B2Granted

Streaming engine for machine learning architecture

AssigneeMarvell Asia Pte Ltd

InventorsAvinash Sodani, Ulf Hanebutte, Senad Durakovic, Hamid Reza Ghasemi, Chia-Hsin Chen

Life of this patent

Estimating the term…

Loading patent…

Abstract

A programmable hardware system for machine learning (ML) includes a core and a streaming engine. The core receives a plurality of commands and a plurality of data from a host to be analyzed and inferred via machine learning. The core transmits a first subset of commands of the plurality of commands that is performance-critical operations and associated data thereof of the plurality of data for efficient processing thereof. The first subset of commands and the associated data are passed through via a function call. The streaming engine is coupled to the core and receives the first subset of commands and the associated data from the core. The streaming engine streams a second subset of commands of the first subset of commands and its associated data to an inference engine by executing a single instruction.

Claims29

Dependent claims

A programmable hardware system for machine learning (ML), comprising:

a core configured to
receive a plurality of commands and data from a host to be analyzed and inferred via machine learning;
divide the plurality of commands into a first subset of commands associated with performance-critical operations and a second subset of commands associated with performance-noncritical operations, wherein the performance-critical operations include at least one or more of a matrix operation, tanh operation, sigmoid operation, memory transpose operation, addition operations, and operations on one or more of any of a tree, a graph, and a priority queue, wherein the performance-noncritical operations include at least one or more of data collection and data mapping, wherein the performance-critical operations exclude any of data collection and data mapping, wherein the performance-noncritical operations exclude any of a matrix operation, tanh operation, sigmoid operation, memory transpose operation, addition operations, and operations on one or more of a tree, a graph, and a priority queue;
transmit each command of the first subset of commands of the plurality of commands for performance-critical operations and associated data thereof to an inference engine for processing via a function call, wherein the each command of the first subset of commands and/or the associated data are encapsulated as parameters in the function call,
wherein the second subset of commands associated with performance-noncritical operations is not transmitted to the inference engine;
an instruction streaming engine coupled to the core and further coupled to the inference engine, wherein the streaming engine is configured to
retrieve and maintain the each command of the first subset of commands and/or the associated data from the function call at a specific location in a buffer;
stream the each command of the first subset of commands and/or its associated data to the inference engine from the buffer;
and
said inference engine configured to
retrieve the each command of the first subset of commands and/or its associated data streamed from the buffer;
perform the performance-critical operations according to the each command of the first subset of commands;
analyze the data;
and
infer a subject from the data.

12 more elements

A programmable hardware system for machine learning (ML), comprising:

a core configured to receive a plurality of commands and a plurality of data from a host to be analyzed and inferred via machine learning, wherein the core is configured to divide the plurality of commands into a first subset of commands associated with performance-critical operations and a second subset of commands associated with performance-noncritical operations, wherein the performance-critical operations include at least one or more of a matrix operation, tanh operation, sigmoid operation, memory transpose operation, addition operations, and operations on one or more of any of a tree, a graph, and a priority queue, wherein the performance-noncritical operations include at least one or more of, data collection and data mapping, wherein the performance-critical operations exclude any of data collection and data mapping, wherein the performance-noncritical operations exclude any of a matrix operation, tanh operation, sigmoid operation, memory transpose operation, addition operations, and operations on one or more of a tree, a graph, and a priority queue and wherein the core is further configured to transmit the first subset of commands of the plurality of commands that is performance-critical operations and associated data thereof of the plurality of data for processing, wherein the first subset of commands and the associated data are passed through via a function call, wherein the second subset of commands associated with performance noncritical operations is not transmitted to the inference engine;
a streaming engine coupled to the core configured to receive the first subset of commands and the associated data from the core, and wherein the streaming engine is configured to stream a third subset of commands of the first subset of commands and its associated data to an inference engine by executing a single instruction;
and
said inference engine configured to
perform the performance-critical operations according to the third set of commands;
analyze the data;
and
infer a subject from the data.

5 more elements

A method comprising:

receive a plurality of commands and data to be analyzed and inferred via machine learning;
dividing the plurality of commands into a first subset of commands associated with performance-critical operations and a second subset of commands associated with performance-noncritical operations, wherein the performance-critical operations include at least one or more of a matrix operation, tanh operation, sigmoid operation, memory transpose operation, addition operations, and operations on one or more of any of a tree, a graph, and a priority queue, wherein the performance-noncritical operations include at least one or more of data collection and data mapping, wherein the performance-critical operations exclude any of data collection and data mapping, wherein the performance-noncritical operations exclude any of a matrix operation, tanh operation, sigmoid operation, memory transpose operation, addition operations, and operations on one or more of a tree, a graph, and a priority queue;
transmitting each command of the first subset of commands of the plurality of commands for performance-critical operations and associated data thereof to an inference engine for processing via a function call, wherein the each command of the first subset of commands and the associated data are encapsulated as parameters in the function call,
wherein the second subset of commands associated with performance noncritical operations is not transmitted to the inference engine;
retrieving and maintaining the each command of the first subset of commands and/or the associated data from the function call at a specific location in a buffer;
streaming the each command of the first subset of commands and/or its associated data to the inference engine from the buffer;
and
retrieving the each command of the first subset of commands and/or its associated data streamed from the buffer;
and
performing the performance-critical operations according to the each command of the first subset of commands;
analyzing the data;
and
inferring a subject from the data.

10 more elements

Dependencies are inferred from the claim text.

Facts

Application: 16/226,534Filed 19 Dec 2018 · us provisional application
Priority: 8 Feb 2018
Published: US 2019/0244117 A18 Aug 2019
Granted: 8 Oct 2024
Term ends: Not estimatedThe source records don’t support an estimate.

Classification17 codes

G06F 9/3879
Using a secondary processor, e.g. coprocessor: for non-native instruction execution, e.g. executing a command; for Java instruction set
G06F 9/30174
Runtime instruction translation, e.g. macros: for non-native instruction set, e.g. Javabyte, legacy code
G06F 9/3836
Concurrent instruction execution, e.g. pipeline or look ahead: instruction issuing, e.g. dynamic instruction scheduling or out of order…
G06F 9/3851
Instruction issuing, e.g. dynamic instruction scheduling or out of order instruction execution: from multiple instruction streams, e.g. multistreaming

Family

Application62/628,130Us provisional application · filed 8 Feb 2018
Application62/644,352Us provisional application · filed 16 Mar 2018
Application62/675,076Us provisional application · filed 22 May 2018
US 2019/0244117 A1Pre-grant publication · published 8 Aug 2019
US 12,112,174 B2This patent

From the US filing history. Foreign family members aren’t listed.