Gaze-based command disambiguation
AssigneeApple Inc.
InventorsKenneth M. KARAKOTSIOS, James BYUN, Pulah J. SHAH
Life of this patent
Estimating the term…
Abstract
Aspects of the subject technology provide improved techniques for human-computer interactions including disambiguation of a human user’s linguistic command. The improved techniques may include integrating linguistic input from a user with additional input from sensors relating to the user and the user’s environment. In an aspect, imagery of a user’s environment may be first analyzed to identify objects in the environment. Input regarding the user, such as a user’s gaze location, may trigger a second analysis of a subset of the identified objects in the environment. The results of these analyses may then be used to resolve an ambiguity in linguistic user input from the user.
Claims15
A method, comprising:
- at an electronic device with a display:
- causing a first sensor of a first external electronic device to scan a proximate area;
- analyzing the scan of the proximate area to identify objects in the proximate area;
- after analyzing the scan of the proximate area, detecting, with a second sensor of a second external electronic device different from the first external electronic device, a first user input corresponding to at least one of the identified objects;
- in response to detecting the first user input:
- transmitting a request for information regarding the at least one of the identified objects;
- and
- receiving, in response to the request, information regarding the at least one of the identified objects;
- receiving a second user input, wherein the second user input includes a command;
- and
- performing the command included in the second user input based at least in part on the information regarding the at least one of the identified objects.
An electronic device comprising:
- a display;
- one or more processors;
- a memory;
- and
- one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for:
- causing a first sensor of a first external electronic device to scan a proximate area;
- analyzing the scan of the proximate area to identify objects in the proximate area;
- after analyzing the scan of the proximate area, detecting, with a second sensor of a second external electronic device different from the first external electronic device, a first user input corresponding to at least one of the identified objects;
- in response to detecting the first user input:
- transmitting a request for information regarding the at least one of the identified objects;
- and
- receiving, in response to the request, information regarding the at least one of the identified objects;
- receiving a second user input, wherein the second user input includes a command;
- and
- performing the command included in the second user input based at least in part on the information regarding the at least one of the identified objects.
A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by one or more processors of an electronic device with a display, cause the electronic device to:
- cause a first sensor of a first external electronic device to scan a proximate area;
- analyze the scan of the proximate area to identify objects in the proximate area;
- after analyzing the scan of the proximate area, detect, with a second sensor of a second external electronic device different from the first external electronic device, a first user input corresponding to at least one of the identified objects;
- in response to detecting the first user input:
- transmit a request for information regarding the at least one of the identified objects;
- and
- receive, in response to the request, information regarding the at least one of the identified objects;
- receive a second user input, wherein the second user input includes a command;
- and
- perform the command included in the second user input based at least in part on the information regarding the at least one of the identified objects.
Dependencies are inferred from the claim text.