Orientation aware smart input correction

A computer-readable storage medium comprising instructions, which, when executed by a processor of a controllable device, cause the controllable device to perform operations comprising:

• receiving a physical input from an input device of the controllable device at a first time, the physical input indicating a physical input direction of a movable member that is movably attached to the input device and generated in response to a movement of the movable member in the physical input direction relative to the input device;
• receiving orientation information from a gyroscope of the input device, the orientation information defining an orientation of the input device at the first time;
• providing the physical input and orientation information to a trained machine learning model, wherein the trained machine learning model is configured to output a corrected input based on the physical input indicating a direction of the movable member and the orientation information defining the orientation of the input device, the corrected input being a prediction of an intended input that is different from the physical input and that indicates a movement of the movable member in an intended input direction relative to the input device and that is different from the physical input direction relative to the input device;
• and
• receiving from the trained machine learning model, the corrected input corresponding to the physical input.

A system comprising one or more processors configured to perform the operations comprising:

• receiving a physical input from an input device of a controllable device at a first time, the physical input indicating a physical input direction of a movable member that is movably attached to the input device and generated in response to a movement of the movable member in the physical input direction relative to the input device;
• receiving orientation information from a gyroscope of the input device, the orientation information defining an orientation of the input device at the first time;
• providing the physical input and orientation information to a trained machine learning model, wherein the trained machine learning model is configured to output a corrected input based on the physical input indicating a direction of the movable member and the orientation information defining the orientation of the input device, the corrected input being a prediction of an intended input that is different from the physical input and that indicates a movement of the movable member in an intended input direction relative to the input device and that is different from the physical input direction relative to the input device;
• and
• receiving from the trained machine learning model, the corrected input corresponding to the physical input.

A computer-implemented method of training a machine learning model to correct a physical input to an input device of a controllable device, the method comprising:

• providing a training dataset comprising a plurality of training instances for an input device that includes a movable member that is movably attached to the input device, each training instance comprising:
• an intended input of the movable member in an intended input direction relative to input device at a particular time and indicating an intended movement of movable member relative to the input device;
• a physical input of the input device at the particular time and indicating a physical movement of movable member relative to the input device;
• and
• orientation information of the input device at the particular time defining an orientation of the input device at the particular time;
• and
• training the machine learning model based on the plurality of training instances to predict the intended input at the particular time based on the physical input and the orientation information to provide a trained machine learning model.