Systems and methods for providing feedback by tracking user gaze and gestures
AssigneeSony Computer Entertainment Inc.
InventorEric J. Larsen
Life of this patent
Estimating the term…
Abstract
User interface technology that provides feedback to the user based on the user's gaze and a secondary user input, such as a hand gesture, is provided. A camera-based tracking system may track the gaze direction of a user to detect which object displayed in the user interface is being viewed. The tracking system also recognizes hand or other body gestures to control the action or motion of that object, using, for example, a separate camera and/or sensor. The user interface is then updated based on the tracked gaze and gesture data to provide the feedback.
Claims35
A computer system comprising:
- a processor configured to receive gaze data, receive gesture data, determine a location of a user interface corresponding to the gaze data and correlate the gesture data to a modification of the user interface;
- and
- memory coupled to the processor and configured to store the gaze data and gesture data.
A system comprising:
- a display to display a user interface that includes an object;
- a gaze sensor to capture eye gaze data;
- a gesture sensor to capture user gesture data;
- and
- a computing device coupled to the gaze sensor, the gesture sensor and the display, wherein the computing device is configured to provide the user interface to the display, determine if the user is viewing the object based on the gaze data, correlate the gesture data to a command corresponding to the object, and modify the display of the user interface that includes the object based on the command.
A method comprising:
- displaying a user interface on a display;
- receiving gaze data for a user interacting with the user interface;
- determining whether the gaze of the user is directed at an object displayed in the user interface based on the gaze data;
- receiving gesture data corresponding to a gesture of the user;
- correlating the gesture data to an intended interaction of the user with the object;
- and
- modifying the display of the object in the user interface based on the correlated interaction.
A computer-readable storage media having computer executable instructions stored thereon which cause a computer system to carry out a method when executed, the method comprising:
- displaying a user interface on a display;
- receiving gaze data for a user interacting with the user interface;
- determining whether the gaze of the user is directed at an object displayed in the user interface based on the gaze data;
- receiving gesture data corresponding to a gesture of the user;
- correlating the gesture data to an intended interaction of the user with the object;
- and
- modifying the display of the object in the user interface based on the correlated interaction.
A computer system comprising:
- a processor configured to receive gaze data, receive at least one of brainwave data and emotion data, determine a location of a user interface corresponding to the gaze data and correlate the at least one of brainwave data and emotion data to a modification of the user interface;
- and
- memory coupled to the processor and configured to store the gaze data and at least one of brainwave data and emotion data.
A system comprising:
- a display to display a user interface that includes an object;
- a gaze sensor to capture eye gaze data;
- a headset comprising a plurality of sensors to capture at least one of brainwave data and emotion data;
- and
- a computing device coupled to the gaze sensor, the headset and the display, wherein the computing device is configured to provide the user interface to the display, determine if the user is viewing the object based on the gaze data, correlate the at least one of brainwave data and emotion data to a command corresponding to the object, and modify the display of the user interface that includes the object based on the command.
A method comprising:
- displaying a user interface on a display;
- receiving gaze data for a user interacting with the user interface;
- determining whether the gaze of the user is directed at an object displayed in the user interface based on the gaze data;
- receiving at least one of brainwave data and emotion data;
- correlating the at least one of brainwave data and emotion data to an intended interaction of the user with the object;
- and
- modifying the display of the object in the user interface based on the correlated interaction.
A computer-readable storage media having computer executable instructions stored thereon which cause a computer system to carry out a method when executed, the method comprising:
- displaying a user interface on a display;
- receiving gaze data for a user interacting with the user interface;
- determining whether the gaze of the user is directed at an object displayed in the user interface based on the gaze data;
- receiving at least one of brainwave data and emotion data;
- correlating the at least one of brainwave data and emotion data to an intended interaction of the user with the object;
- and
- modifying the display of the object in the user interface based on the correlated interaction.
Dependencies are inferred from the claim text.