Multiple core processor for efficient inference of machine learning workloads

A method comprising:

• sending, by a power management circuit, a first set of configuration values that configures a plurality of processor cores, an interconnect, and a plurality of cache coherence circuits that manage access on the interconnect between the plurality of processor cores and a cache into a first mode to generate a first output token from a machine learning model;
• generating, by the plurality of processor cores, the interconnect, and the plurality of cache coherence circuits in the first mode, the first output token from the machine learning model;
• sending, by the power management circuit, a second set of configuration values that reconfigures the plurality of processor cores, the interconnect, and the plurality of cache coherence circuits into a second mode to generate a second output token from the machine learning model;
• and
• generating, by the plurality of processor cores, the interconnect, and the plurality of cache coherence circuits in the second mode, the second output token from the machine learning model for an input of the first output token.

A system comprising:

• a system memory;
• a cache;
• a plurality of processor cores;
• an interconnect;
• a plurality of cache coherence circuits to manage access on the interconnect between the plurality of processor cores, the cache, and the system memory;
• and
• a power management circuit to cause a switch from a first mode that configures the plurality of processor cores, the interconnect, and the plurality of cache coherence circuits to generate a first output token from a machine learning model to a second mode that reconfigures a subset of the plurality of processor cores, the interconnect, and the plurality of cache coherence circuits to generate at least a second output token from the machine learning model.