Back to results
US 2023/0400918 A1Published application
Systems and Methods for Hands-Free Scrolling Based on a Detected User Reading Activity
AssigneeAthena Accessible Technology, Inc.
InventorsBlake Francis, Stephen Mattison, David Chiu
Life of this patent
Estimating the term…
Abstract
Systems and methods for hands-free scrolling are disclosed herein. An example method includes detecting an eye movement or gaze of a user relative to content displayed on a display screen of the computing device, the content being arranged along a y axis of the display screen, determining that the user has viewed a portion of the content provided on the display screen and where along the y axis the eye movement or gaze is directed; and automatically scrolling the content on the display in response to the eye movement or gaze of the user in relation to the y axis.
Claims20
A method comprising:
- detecting, via an eye tracking device of a computing device, eye gaze data comprising a plurality of eye gaze data points of an eye movement of a user, the plurality of eye gaze data points comprising X, Y, and Z eye gaze coordinates;
- sampling the detected eye gaze data points at a prescribed sampling interval;
- collecting the sampled eye gaze data points into window periods having a prescribed window size;
- calculating, for the sampled eye gaze data points in each window period, a weighted average of the X eye gaze coordinates;
- determining, by a first machine learning model, for the sampled eye gaze data points in each window period,
- a) a first probability of determining a user reading activity in each window period based on the calculated weighted average of the X eye gaze coordinates of the sampled eye gaze data points in each window period;
- and
- b) a reading location of weighted Y eye gaze coordinates;
- c) optionally, a mean value of the Z eye gaze coordinates in each window period;
- calculating one or more feature extraction parameters from the sampled eye gaze data points sampled in each window period;
- and
- determining, by a second machine learning model, a second probability of determining a user reading activity in each window period, the second probability being calculated in accordance with the first probability and the weighted Y and mean Z eye gaze coordinates output from the first machine learning model and one or more feature extraction parameters.
9 more elements
A method comprising:
- detecting, via an eye tracking device of a computing device, eye gaze data comprising a plurality of eye gaze data points of an eye movement of a user relative to content displayed on a display screen, the plurality of eye gaze data points comprising X, Y, and Z eye gaze coordinates;
- sampling the detected eye gaze data points at a prescribed sampling interval;
- collecting the sampled eye gaze data points into window periods having a prescribed window size, the eye gaze data points including a plurality of face projection points of a user's face projected onto a display screen of a display device;
- determining if the user is viewing the display screen of the display device based on the positioning of the face projection points;
- in the case where it is determined that the user is viewing the display screen,
- calculating a current mean value of the Y coordinate in a window period;
- calculating a delta Y value of the Y coordinate in the window period;
- calculate a median of the Y coordinates in the window period;
- determine if the delta Y value of the Y coordinate in the window period is greater than twice the line height of content being viewed on the display screen;
- in the case where it is determined that the delta Y value of the Y coordinate in the window period is greater than twice the line height of content being viewed on the display screen, a screen text is automatically scrolled by an offset calculated as:
- Offset=Mean (Y) of Current Reading−Mean (Y) of Previous−Line Height.
8 more elements
A method for predicting an eye gesture, comprising:
- detecting, via an eye tracking device of a computing device, eye gaze data comprising a plurality of eye gaze data points of an eye movement of a user, the plurality of eye gaze data points comprising X, Y, and Z eye gaze coordinates;
- sampling the detected eye gaze data points at a prescribed sampling interval;
- collecting the sampled eye gaze data points into window periods having a prescribed window size, the eye gaze data points including a plurality of face projection points of a user's face viewing content on a display screen of a display device;
- extracting feature data from the X, Y and Z eye gaze coordinates;
- detecting, via a machine learning model, a trigger gesture made by the user;
- in the case where a trigger gesture is detected:
- storing the trigger gesture and state in a historical eye gesture data repository;
- determining if a subsequent eye gesture is detected;
- in the case where a subsequent eye gesture is detected,
- outputting from the machine learning module, an eye gesture class, a probability of the eye gesture class being detected, and a timestamp;
- and
- storing the eye gesture in the historical eye gesture data repository.
9 more elements
A system comprising:
- a display;
- an eye tracking device;
- and
- a controller comprising:
- a memory;
- and
- a processor communicatively coupled to the memory the memory storing instructions executable by the processor to:
- detect, via an eye tracking device of a computing device, eye gaze data comprising a plurality of eye gaze data points of an eye movement of a user relative to content displayed on a display screen, the plurality of eye gaze data points comprising X, Y, and Z eye gaze coordinates;
- sample the detected eye gaze data points at a prescribed sampling interval;
- collect the sampled eye gaze data points into window periods having a prescribed window size;
- calculate, for the sampled eye gaze data points in each window period, a weighted average of the X eye gaze coordinates;
- determine, by a first machine learning model, for the sampled eye gaze data points in each window period,
- a) a first probability of the user reading the text on the display device based on the calculated weighted average of the X eye gaze coordinates of the sampled eye gaze data points;
- and
- b) a reading location of the Y and Z eye gaze coordinates;
- calculate a plurality of feature extraction parameters from the sampled eye gaze data points collected at each window period;
- and
- determine, by a second machine learning model, a second probability of the user reading the text on the display device in each window period based on the first probability of the user reading text on the display device, the reading location of the Y and Z eye gaze coordinates and the calculated plurality of feature extraction parameters.
15 more elements
Dependencies are inferred from the claim text.