Back to results
US 12,591,297 B2Granted
Multimodal task execution and text editing for a wearable system
AssigneeMagic Leap, Inc.
InventorsJames M. Powderly, Savannah Niles, Jennifer M.R. Devine, Adam C. Carlson, Jeffrey Scott Sommers, Praveen Babu J D, Ajoy Savio Fernandes, Anthony Robert Sheeder
Life of this patent
Estimating the term…
Abstract
Examples of wearable systems and methods can use multiple inputs (e.g., gesture, head pose, eye gaze, voice, and/or environmental factors (e.g., location)) to determine a command that should be executed and objects in the three-dimensional (3D) environment that should be operated on. The multiple inputs can also be used by the wearable system to permit a user to interact with text, such as, e.g., composing, selecting, or editing text.
Claims19
A method for interacting with virtual content, comprising:
- receiving a first direct user input corresponding to a first virtual object and associating the first direct user input with a first weight;
- determining a first confidence score correlated to the first direct user input, wherein the first confidence score is based, at least in part, on a first authentication level associated with the first direct user input;
- receiving a second direct user input, corresponding to a second virtual object and having a different modality than the first direct user input, and associating the second direct user input with a second weight;
- determining a second confidence score correlated to the second direct user input, wherein the second confidence score is based, at least in part, on a second authentication level associated with the second direct user input, the second authentication level being different than the first authentication level;
- calculating an aggregate confidence score from at least the first confidence score, the first weight, the second confidence score, and the second weight;
- identifying a subject virtual object from at least the first virtual object and the second virtual object based on the aggregate confidence score;
- and
- manipulating the subject virtual object to meet a parameter condition.
5 more elements
A method for interacting with virtual content, comprising:
- receiving a direct user input corresponding to a first virtual object and associating the direct user input with a first weight;
- determining a first confidence score correlated to the direct user input;
- receiving an indirect user input corresponding to a second virtual object and associating the indirect user input with a second weight;
- determining a second confidence score correlated to the indirect user input;
- calculating an aggregate confidence score from at least the first confidence score, the first weight, the second confidence score, and the second weight;
- identifying a subject virtual object from at least the first virtual object and the second virtual object based on the aggregate confidence score;
- and
- manipulating the subject virtual object to meet a parameter condition.
5 more elements
A system for interacting with objects for a wearable device, the system comprising:
- a display system of a wearable device configured to present a three-dimensional (3D) view to a user and permit a user interaction with objects in a field of regard (FOR) of the user, the FOR comprising a portion of an environment around the user that is capable of being perceived by the user via the display system;
- a hardware processor in communication with a sensor and the display system, the hardware processor programmed to:
- receive multimodal inputs for the user interaction;
- parse the multimodal inputs to identify a subject, a command, and a parameter of the user interaction;
- and
- execute the user interaction based on the subject, the command, and the parameter;
- wherein to identify the subject of the user interaction, the wearable device is programmed to:
- access a first value to a first input of the multimodal inputs;
- access a second value to a second input of the multimodal inputs;
- calculate a first confidence score for a first candidate virtual object based on the first value or the second value;
- calculate a second confidence score for a second candidate virtual object based on the first value or the second value;
- and set the subject to be either the first candidate virtual object or the second candidate virtual object based on a comparison of the first confidence score and the second confidence score such that the subject is the first candidate virtual object or the second candidate virtual object with a higher confidence score,
- wherein the first input is associated with a first weight and the second input is associated with a second weight, and wherein the first value and the second value are assigned based on the first weight and the second weight respectively.
10 more elements
Dependencies are inferred from the claim text.