System-on-chip having non-volatile memory storing a machine learning model and performing inference using the model

A computing device comprising,

• a volatile memory;
• a non-volatile memory storing a machine learning (ML) model;
• and
• one or more processers adapted to:
• read a set of values of the ML model;
• apply the set of values of the ML model to input data;
• based on applying the set of values to the input data, determine regions of the ML model having fixed values and regions of the ML model having non-fixed values;
• and
• migrate or store the regions of the ML model having non-fixed values in the volatile memory of the computing device.

A system-on-chip (SoC) for vehicle operations, comprising:

• a set of chiplets that includes DRAM;
• a flash memory chiplet having flash memory adapted to store a machine learning (ML) model, the flash memory having a storage capacity of at least 10 GB, the ML model including model weights that have been trained;
• and
• an interconnect to connect the flash memory to the set of chiplets;
• wherein the set of chiplets, when integrated into a vehicle, are adapted to
• receive an input prompt generated based on vehicle sensor data generated by the vehicle, passenger input from a passenger of the vehicle, or a combination thereof;
• apply the ML model to the input prompt to perform inference by:
• accessing a first set of the model weights of the ML model directly from the flash memory to calculate intermediate values based on the input prompt and the first set of model weights;
• storing the intermediate values in the DRAM;
• accessing a second set of the model weights of the ML model directly from the flash memory to calculate an output of the ML model based on the intermediate values and the second set of model weights;
• storing the output of the ML model in the DRAM;
• and
• generate a vehicle action based on the output of the ML model.