Training neural network model and performing inference using heterogeneous processors

A system for generating a neural network model, comprising:

• a first processing circuit configured to:
• train or retrain a neural network model to obtain first weights of the neural network model,
• receive evaluation results,
• retrain the neural network model based on the received evaluation results to modify a structure of the neural network model and to update the first weights, and
• send the updated first weights associated with an updated neural network model with the modified structure, and
• a second processing circuit having a configuration different from the first processing circuit and with limited capability relative to the processing circuit, the second processing circuit configured to:
• receive the first weights,
• perform inference using the neural network model and the received first weights,
• generate inference results and the evaluation results on the performed inference, responsive to performing the inference using the neural network model and the received first weights,
• send the evaluation results to the first processing circuit,
• receive the updated first weights associated with the updated neural network model from the first processing circuit, the updated first weights and the updated neural network model configured to be executed on the second processing circuit, and
• perform inference using the updated neural network model and the updated first weights.

A non-transitory computer readable storage medium in a neural processing unit storing updated weights of a neural network model thereon, the weights generated by:

• obtaining previous weights of the neural network model by training or retraining the neural network model on a general-purpose graphics processing unit (GPGPU);
• transferring the previous weights to the neural processing unit with limited capability relative to the GPGPU;
• performing inference by the neural processing unit using neural network model and the transferred previous weights;
• generating inference results and evaluation results on the performance of the inference at the neural processing unit using the neural network and the transferred previous weights, responsive to performing the inference by the neural processing unit;
• sending the evaluation results to the GPGPU to modify a structure of the neural network model and to update the previous weights into updated weights by the GPGPU for execution on the neural processing unit;
• sending the updated weights associated with an updated neural network model with the modified structure to the neural processing unit to the neural processing unit;
• and
• performing inference by the neural processing unit using the updated neural network model and the updated weights.