Back to results
US 11,237,623 B2Granted
Multimodal task execution and text editing for a wearable system
AssigneeMagic Leap, Inc.
InventorsJames M. Powderly, Savannah Niles, Jennifer M. R. Devine, Adam C. Carlson, Jeffrey Scott Sommers, Praveen Babu J D
Life of this patent
Estimating the term…
Abstract
Examples of wearable systems and methods can use multiple inputs (e.g., gesture, head pose, eye gaze, voice, and/or environmental factors (e.g., location)) to determine a command that should be executed and objects in the three-dimensional (3D) environment that should be operated on. The multiple inputs can also be used by the wearable system to permit a user to interact with text, such as, e.g., composing, selecting, or editing text.
Claims20
A system comprising:
- An outward facing imaging system of a wearable system configured to acquire first user input data from a user in a visual mode of input;
- a second sensor of the wearable system configured to acquire second user input data in a second mode of input, the second mode of input different from the visual mode of input;
- and
- a hardware processor in communication with the outward facing imaging system and the second sensor, the hardware processor programmed to:
- receive the first user input data from the outward facing imaging system;
- receive the second user input data from the second sensor;
- determine, based on the first user input data, a first proportion of a first candidate object of a plurality of candidate objects that is in a field of view of the user with respect to a second proportion of the first candidate object that is outside of the field of view of the user;
- determine a first confidence score associated with the first candidate object based on the first proportion and the second proportion;
- determine a second confidence score for the first candidate object based on the second user input data;
- and
- cause a user interface operation to be performed on the first candidate object based at least in part on the first confidence score and the second confidence score.
8 more elements
A method comprising:
- under control of a hardware processor of a wearable system in communication with a plurality of sensors configured to acquire user input data:
- receiving a first user input data from an outward facing imaging system;
- receiving a second user input data from a second sensor, the second user input data having a second input mode different from an input mode of the first user input data;
- determining, based on the first user input data, a first proportion of a first candidate object of a plurality of candidate objects that is in a field of view of a user with respect to a second proportion of the first candidate object that is outside of the field of view of the user;
- determining a first confidence score associated with the first candidate object based on the first proportion and the second proportion;
- determining a second confidence score for the first candidate object based on the second user input data;
- and
- causing a user interface operation to be performed on the first candidate object based at least in part on the first confidence score and the second confidence score.
5 more elements
Dependencies are inferred from the claim text.