Systems and methods for gesture inference using transformations
AssigneePison Technology, Inc.
InventorsDexter Ang, David Cipoletta, Xiaofeng Tan, Matt Fleury, Dylan Pollack
Life of this patent
Estimating the term…
Abstract
Disclosed are methods, systems and non-transitory computer readable memory for gesture inference. For instance, a first method may include computer vision to train and/or infer gesture inferences. For instance, a second method may include using transformations to data and/or ML models to address inter/intra-session variability of sensor data. For instance, a third method may include using ML model selection to select a ML model to address inter/intra-session variability of sensor data.
Claims20
A system for gesture inference, the system comprising:
- a wearable device configured to be worn on a portion of an arm of a user, the wearable device comprising:
- a biopotential sensor, the biopotential sensor being configured to obtain biopotential data indicating electrical signals generated by nerves and muscles in the arm of the user;
- and
- a motion sensor, the motion sensor being configured to obtain motion data relating to a motion of the portion of the arm of the user, the motion data and biopotential data collectively being sensor data;
- and
- a processing pipeline configured to receive the biopotential data and the motion data and process the biopotential data and the motion data to generate a gesture inference output using a ML model, wherein the processing pipeline includes:
- a pre-process module configured to:
- obtain a first set of sensor data;
- determine, based on the sensor data or a derivative thereof, a first transformation to the ML model and/or a second transformation to the first set of sensor data;
- and
- apply the first transformation to the ML model to obtain a session ML model and/or apply the second transformation to the first set of sensor data or derivative thereof to obtain mapped sensor data;
- and
- an inference module configured to infer the gesture inference based on (1) the session ML model and the first set of sensor data, and/or (2) the ML model and the mapped sensor data;
- wherein the system is configured to, based on the gesture inference, determine a machine interpretable event, and execute an action corresponding to the machine interpretable event.
A computer-implemented method for gesture inference, the computer-implemented method comprising:
- obtaining a first set of sensor data, the first set of sensor data including motion data and biopotential data, the motion data being obtained by a motion sensor of a wearable device, the motion sensor being configured to obtain the motion data relating to a motion of a portion of an arm of a user wearing the wearable device, the biopotential data being obtained by a biopotential sensor of the wearable device, the biopotential sensor being configured to obtain the biopotential data indicating electrical signals generated by nerves and muscles in the arm of the user;
- determining, based on the sensor data or a derivative thereof, a first transformation to a ML model and/or a second transformation to the first set of sensor data;
- applying the first transformation to the ML model to obtain a session ML model and/or applying the second transformation to the first set of sensor data or derivative thereof to obtain mapped sensor data;
- and
- inferring a gesture inference based on (1) the session ML model and the first set of sensor data, and/or (2) the ML model and the mapped sensor data;
- wherein the wearable device is configured to, based on the gesture inference, determine a machine interpretable event, and execute an action corresponding to the machine interpretable event.
Dependencies are inferred from the claim text.