US 10,013,053 B2 · System for gaze interaction

Abstract

A method and system for assisting a user when interacting with a graphical user interface combines gaze based input with gesture based user commands. A user of a computer system without a traditional touch-screen can interact with graphical user interfaces in a touch-screen like manner using a combination of gaze based input and gesture based user commands. A solution for touch-screen like interaction uses gaze input and gesture based input as a complement or an alternative to touch-screen interactions with a computer device having a touch-screen. Combined gaze and gesture based interaction with graphical user interfaces can be used to achieve a touchscreen like environment in computer systems without a traditional touchscreen or in computer systems having a touchscreen arranged ergonomically unfavorable for the user or a touchscreen arranged such that it is more comfortable for the user to use gesture and gaze for the interaction than the touchscreen.

Claims8

Dependent claims

A control module for executing object selection commands during user interaction with an information presentation area displaying an object, wherein the control module is configured to:

acquire user gaze data signals from a gaze tracking module, the user gaze signals indicating a gaze point on the information presentation area;
receive, from an input device that includes a physical touch sensitive area, first user gesture data corresponding to a press of a finger on the physical touch sensitive area;
generate a gaze point area on the information presentation area based on the user gaze data signals and on the first user gesture data, the gaze point area being a local area that includes the gaze point;
receive, from the input device, second user gesture data corresponding to a movement of the finger on the physical touch sensitive area, the press of the finger followed next by the movement of the finger;
determine, based on the gaze point area, that the second user gesture data is associated with a fine tuning command to update the gaze point area;
resize, based on the fine tuning command, the gaze point area according to the second user gesture data to generate an updated gaze point area, the updated gaze point area including the object and overlapping with the gaze point area;
receive, from the input device, third user gesture data indicating a release of the finger from the physical touch sensitive area, the movement of the finger followed next by the release of the finger;
determine, based on the object being included in the updated gaze point area, that the third user gesture data is associated with a select command to select the object;
and
select, based on the select command, the object in the updated gaze point area.

7 more elements

A handheld portable device including an information presentation area and comprising an input device comprising a physical touch sensitive area, said user input device being adapted to detect user gestures in a form of user finger touch on, movement on and release from the physical touch sensitive area, and a gaze tracking module adapted to detect gaze data of a user of said information presentation area, said handheld portable device further comprising a control module configured to:

acquire user gaze data signals from the gaze tracking module, the user gaze signals indicating a gaze point on the information presentation area;
receive, from the input device, first user gesture data corresponding to a press of a finger on the physical touch sensitive area;
generate a gaze point area on the information presentation area based on the user gaze data signals and on the first user gesture data, the gaze point area being a local area that includes the gaze point;
receive, from the input device, second user gesture data corresponding to a movement of the finger on the physical touch sensitive area, the press of the finger followed next by the movement of the finger;
determine, based on the gaze point area, that the second user gesture data is associated with a fine tuning command to update the gaze point area;
resize, based on the fine tuning command, the gaze point area according to the second user gesture data to generate an updated gaze point area, the updated gaze point area including the object and overlapping with the gaze point area;
receive, from the input device, third user gesture data indicating a release of the finger from the physical touch sensitive area, the movement of the finger followed next by the release of the finger;
determine, based on the object being included in the updated gaze point area, that the third user gesture data is associated with a select command to select the object;
and
select, based on the select command, the object in the updated gaze point area.

7 more elements

A system for user interaction with an information presentation area, said system comprising:

an input device adapted to detect user gestures, said input device comprising at least one physical touch sensitive area arranged on a steering device of a vehicle or adapted to be integrated in a steering device of a vehicle, said touchpad being adapted to detect user gestures in a form of user finger touch on, movement on and release from the physical touch sensitive area;
a gaze tracking module adapted to detect gaze data of a user of said information presentation area;
a control module configured to:
acquire user gaze data signals from the gaze tracking module, the user gaze signals indicating a gaze point on the information presentation area;
receive, from the input device, first user gesture data corresponding to a press of a finger on the physical touch sensitive area;
generate a gaze point area on the information presentation area based on the user gaze data signals and on the first user gesture data, the gaze point area being a local area that includes the gaze point;
receive, from the input device, second user gesture data corresponding to a movement of the finger on the physical touch sensitive area, the press of the finger followed next by the movement of the finger;
determine, based on the gaze point area, that the second user gesture data is associated with a fine tuning command to update the gaze point area;
resize, based on the fine tuning command, the gaze point area according to the second user gesture data to generate an updated gaze point area, the updated gaze point area including the object and overlapping with the gaze point area;
receive, from the input device, third user gesture data indicating a release of the finger from the physical touch sensitive area, the movement of the finger followed next by the release of the finger;
determine, based on the object being included in the updated gaze point area, that the third user gesture data is associated with a select command to select the object;
and
select, based on the select command, the object in the updated gaze point area.

10 more elements

A wireless transmit/receive unit, WTRU, associated with an information presentation area and comprising an input device comprising a touch sensitive area and being adapted to detect user gestures in a form of user finger touch on, movement on and release from the touch sensitive area, and a gaze tracking module adapted to detect gaze data of a viewer of said information presentation area, said WTRU further comprising a control module configured to:

acquire user gaze data signals from the gaze tracking module, the user gaze signals indicating a gaze point on the information presentation area;
receive, from the input device, first user gesture data corresponding to a press of a finger on the physical touch sensitive area;
generate a gaze point area on the information presentation area based on the user gaze data signals and on the first user gesture data, the gaze point area being a local area that includes the gaze point;
receive, from the input device, second user gesture data corresponding to a movement of the finger on the physical touch sensitive area, the press of the finger followed next by the movement of the finger;
determine, based on the gaze point area, that the second user gesture data is associated with a fine tuning command to update the gaze point area;
resize, based on the fine tuning command, the gaze point area according to the second user gesture data to generate an updated gaze point area, the updated gaze point area including the object and overlapping with the gaze point area;
receive, from the input device, third user gesture data indicating a release of the finger from the physical touch sensitive area, the movement of the finger followed next by the release of the finger;
determine, based on the object being included in the updated gaze point area, that the third user gesture data is associated with a select command to select the object;
and
select, based on the select command, the object in the updated gaze point area.

7 more elements

Dependencies are inferred from the claim text.

System for gaze interaction

Life of this patent

System for gaze interaction

Life of this patent

Abstract

Claims8