Gaze-based command disambiguation
AssigneeApple Inc.
InventorsKenneth M. Karakotsios, James Byun, Pulah J. Shah
Life of this patent
Estimating the term…
Abstract
Aspects of the subject technology provide improved techniques for human-computer interactions including disambiguation of a human user's linguistic command. The improved techniques may include integrating linguistic input from a user with additional input from sensors relating to the user and the user's environment. In an aspect, imagery of a user's environment may be first analyzed to identify objects in the environment. Input regarding the user, such as a user's gaze location, may trigger a second analysis of a subset of the identified objects in the environment. The results of these analyses may then be used to resolve an ambiguity in linguistic user input from the user.
Claims39
A method, comprising:
- scanning a proximate area with at least one sensor;
- first analyzing the scan of the proximate area to identify objects in the proximate area;
- after first analyzing the scan of the proximate area, detecting a trigger corresponding to at least one of the identified objects based on a first user input with respect to the at least one of the identified objects;
- in response to detecting the trigger;
- transmitting a request for first information regarding the at least one of the identified objects;
- and
- receiving, in response to the request, information regarding the at least one of the identified objects;
- receiving a command from a user, wherein the command includes an ambiguity;
- resolving the ambiguity in the command based on the information regarding the at least one of the identified objects;
- and
- performing the command based at least in part on the resolved ambiguity.
A device comprising:
- a memory;
- and
- at least one processor configured to:
- scan a proximate area with at least one sensor;
- first analyze the scan of the proximate area to identify objects in the proximate area;
- after first analyzing the scan of the proximate area, detect a trigger corresponding to at least one of the identified objects based on a first user input with respect to the at least one of the identified objects;
- in response to detecting the trigger:
- transmit a request for information regarding the at least one of the identified objects;
- and
- receive, in response to the request, information regarding the at least one of the identified objects;
- receive a command from a user, wherein the command includes an ambiguity;
- resolve the ambiguity in the command based on the information regarding the at least one of the identified objects;
- and
- perform the command based at least in part on the resolved ambiguity.
A non-transitory computer readable memory including instructions that, when executed by a processor, cause the processor to:
- scan a proximate area with at least one sensor;
- first analyze the scan of the proximate area to identify objects in the proximate area;
- after first analyzing the scan of the proximate area, detect a trigger corresponding to at least one of the identified objects based on a first user input with respect to the at least one of the identified objects;
- in response to detecting the trigger;
- transmit a request for information regarding the at least one of the identified objects;
- and
- receive, in response to the request, information regarding the at least one of the identified objects;
- receive a command from a user, wherein the command includes an ambiguity;
- resolve the ambiguity in the command based on the information regarding the at least one of the identified objects;
- and
- perform the command based at least in part on the resolved ambiguity.
Dependencies are inferred from the claim text.