Federated on-sensor learning for local device adaptation and global improvement of machine learning model

A non-transitory computer readable storage medium storing instructions that when executed by a computer processor, cause the computer processor to perform steps comprising:

• receiving, by a smart sensor of an artificial reality headset, parameters of a machine learning model;
• updating, by the smart sensor, the machine learning model based on feature data extracted from the sensor data;
• and
• repeatedly performing:
• sending, by the smart sensor, encrypted data associated with the machine learning model to a system on chip (SoC) of the headset;
• receiving, by the SoC, encrypted data associated with the machine learning model from one or more smart sensors;
• sending, by the SoC, encrypted data associated with the machine learning model collected from the one or more smart sensors to a server;
• receiving, by the SoC, parameters of a base machine learning model from the server, and
• sending, by the SoC, parameters of the base machine learning model to the smart sensor;
• and
• locally updating, by the smart sensor, the received base machine learning model based on the stored feature data.