Embedding a state space model on models-on-silicon hardware architecture

An apparatus, comprising:

• a processing circuit to receive input data and generate one or more input tokens;
• and
• an inferencing circuit embedding a neural network, the inferencing circuit to receive the one or more input tokens and output one or more output tokens to the processing circuit, the inferencing circuit comprising:
• a sequential read memory to store one or more parameters of a selective state space model of the neural network;
• a memory to store a state of the selective state space model;
• and
• one or more circuits to perform one or more corresponding operations of the selective state space model based on the state, the one or more parameters in the sequential read memory, and an input to the selective state space model.

A method, comprising:

• reading one or more parameters of a selective state space model of a neural network from a sequential read memory;
• and
• computing, using one or more embedded circuits corresponding to one or more operations of the selective state space model, an output of the selective state space model based on the one or more parameters and an input to the selective state space model, wherein computing the output comprises:
• reading a previous state of the selective state space model from a memory;
• and
• storing a state of the selective state space model in the memory.