Systems and methods for programmable hardware architecture for machine learning

A method to support machine learning (ML) via a programmable hardware system, comprising:

• receiving data from a host to be analyzed and/or inferred via machine learning;
• receiving a plurality of commands;
• coordinating components in the system to perform a ML operation on the data based on the plurality of commands;
• interpreting and dividing the plurality of commands into a first set of commands for performance-critical operations and a second set of commands for performance non-critical operations of the ML operation;
• executing the second set of commands on a core to perform the performance non-critical operations of the ML operation;
• streaming a first set of programming instructions to an inference engine based on the first set of commands;
• generating and transmitting one or more streams of data from a memory to be analyzed and/or inferred by the inference engine;
• and
• processing the one or more streams of data based on the first set of programming instructions, at the inference engine.

A programmable hardware system for machine learning (ML), comprising:

• a memory configured to receive data from a host to be analyzed and/or inferred via machine learning;
• a core configured to receive a plurality of commands from the host and wherein the core is further configured to coordinate components in the system to perform a ML operation on the data, wherein said core is further configured to interpret and divide the plurality of commands received from the host into a first set of commands for performance-critical operations and a second set of commands for performance non-critical operations of the ML operation, and wherein the core is further configured to execute the second set of commands to perform the performance non-critical operations of the ML operation and wherein the core is further configured to provide the first set of commands for the performance-critical operations to be performed by an inference engine via an instruction-streaming engine;
• said instruction-streaming engine coupled to the core, a data streaming engine, and the inference engine, wherein the instruction-streaming engine is configured to stream a first set of programming instructions to the inference engine based on the second set of commands to perform the performance-critical operations;
• said data steaming engine coupled to the inference engine and configured to generate one or more streams of data from the memory to be analyzed and/or inferred by the inference engine;
• and
• said inference engine configured to
• receive the data and the first set of programming instructions from the data streaming engine and the instruction-streaming engine, respectively;
• perform the first set of commands for performance-critical operations of the ML operation by executing the first set of programming instructions to analyze the streams of data received from the data streaming engine.