Gaze-controlled user interface with multimodal input

A personal computer system providing a gaze-controlled graphical user interface, the personal computer system comprising:

• a gaze tracking system operable to provide gaze point data of a viewer of a visual display;
• and
• a processor configured to generate a display signal to be supplied to the visual display, the visual display associated with a reference coordinate system that is stationary with respect to the visual display;
• wherein the personal computer system is configured to:
• determine a gaze point on the visual display based on the gaze point data, the gaze point determined with respect to the reference coordinate system,
• operate in a bidirectional interaction mode, in which the visual display shows multiple graphical controls in motion with respect to the reference coordinate system, wherein each one of the multiple graphical controls in motion, when selected using gaze input, causes a respective predefined input operation to occur,
• operate in a unidirectional interaction mode, in which the visual display shows the multiple graphical controls in constant locations with respect to the reference coordinate system, when selected using a non-gaze input causes the respective predefined input operation to occur;
• transition from the unidirectional interaction mode to the bidirectional interaction mode, wherein the transition causes the multiple graphical controls to be in motion;
• in response to transitioning to the bidirectional interaction mode:
• present at least in a neighborhood of the gaze point a first graphical control in motion and a second graphical control in motion, wherein the neighborhood is substantially centered on the gaze point and is smaller than an entirety of the visual display,
• determine a first path including a first relative motion of the first graphical control and a second path including a second relative motion of the second graphical control, wherein the second relative motion is distinguishable from the first relative motion,
• determine a gaze motion based on relative movements of the gaze point,
• compare the gaze motion to each of the first path and the second path, and
• in response to determining a match between the gaze motion and either the first path or the second, indicate the respective predefined input operation associated with the matching one of the first graphical control or the second graphical control, and
• transition from the bidirectional interaction mode to the unidirectional interaction mode when one or more criteria are met, causing the multiple graphical controls to be in constant locations.