# US 2023/0069764 A1 · Systems and methods for using natural gaze dynamics to detect input recognition errors > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2023/0069764 A1 - Document ID: US20230069764A1 - Status: Published application - Kind: A1 - Application number: 17/866,179 - Filed: 2022-07-15 - Published: 2023-03-02 - Earliest claimed: 2021-08-24 - Assignees: Meta Platforms Technologies, LLC - Applicants: Meta Platforms Technologies, LLC - Inventors: Tanya Renee Jonker; Candace Peacock; Benjamin Lafreniere; Ting Zhang; Hrvoje Benko; Rawan Alghofaili; Michael Glueck - Web page: https://patentgrep.com/patent/US20230069764A1 ## CPC - G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)} - G02B 27/0093: {with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking} - G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) - G06F 9/451: Execution arrangements for user interfaces - G02B 2027/0138: {comprising image capture systems, e.g. camera} - G02B 2027/014: {comprising information/image processing systems} - G06N 20/00: Machine learning - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract A disclosed computer-implemented method may include (1) tracking a gaze of a user as the user interacts with a user interface, (2) determining, based on tracking of the gaze of the user, that a detected user interaction with the user interface represents a false positive input inference by the user interface, and (3) executing at least one remedial action based on determining that the detected user interaction represents the false positive input inference by the user interface. Various other methods, systems, and computer-readable media are also disclosed. ## Claims ### Claim 1 A computer-implemented method comprising: tracking a gaze of a user as the user interacts with a user interface; determining, based on tracking of the gaze of the user, that a detected user interaction with the user interface represents a false positive input inference by the user interface; and executing at least one remedial action based on determining that the detected user interaction represents the false positive input inference by the user interface. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, wherein tracking the gaze of the user comprises extracting at least one gaze feature from the gaze of the user as the user interacts with the user interface. ### Claim 3 (depends on 2) The computer-implemented method of claim 2, wherein the at least one gaze feature comprises at least one of: a fixation duration; an angular displacement between an initial fixation centroid and a subsequent fixation centroid; an angular displacement between an initial saccade centroid and a subsequent saccade centroid; an angular displacement between an initial saccade landing point and a subsequent saccade landing point; an amplitude of a saccade; a duration of a saccade; a fixation probability; a saccade probability; a gaze velocity; or a gaze dispersion. ### Claim 4 (depends on 1) The computer-implemented method of claim 1, wherein determining, based on tracking of the gaze of the user, that the detected user interaction with the user interface represents the false positive input inference by the user interface comprises: training, using gaze features of the user, a machine learning model to discriminate between true positive events and false positive events; and analyzing the tracked gaze of the user using the trained machine learning model. ### Claim 5 (depends on 1) The computer-implemented method of claim 1, wherein determining, based on tracking of the gaze of the user, that the detected user interaction with the user interface represents the false positive input inference by the user interface comprises: training, using gaze features of a group of users, a machine learning model to discriminate between true positive events and false positive events; and analyzing the tracked gaze of the user using the trained machine learning model. ### Claim 6 (depends on 1) The computer-implemented method of claim 1, wherein: executing the at least one remedial action comprises receiving, via the user interface, user input associated with the false positive input inference; and the method further comprises determining, based on additional tracking of the gaze of the user and the user input associated with the false positive input inference, that an additional detected user interaction with the user interface represents an additional false positive input inference by the user interface. ### Claim 7 (depends on 1) The computer-implemented method of claim 1, wherein executing the at least one remedial action comprises: determining that the detected user interaction with the user interface caused a change in an application state of an application associated with the user interface; and automatically undoing the change in the application state. ### Claim 8 (depends on 1) The computer-implemented method of claim 1, wherein executing the at least one remedial action comprises presenting a notification within the user interface that indicates that a false positive input inference has occurred. ### Claim 9 (depends on 8) The computer-implemented method of claim 8, wherein the notification further indicates that the detected user interaction caused a change in an application state of an application associated with the user interface. ### Claim 10 (depends on 8) The computer-implemented method of claim 8, wherein the notification further comprises a confirmation control that enables the user to confirm the detected user interaction. ### Claim 11 (depends on 8) The computer-implemented method of claim 8, wherein: the notification comprises an undo control; and the method further comprises: receiving, via the undo control of the user interface, an instruction to undo a command executed as a result of the detected user interaction; and undoing, in response to receiving the instruction to undo the command executed as a result of the detected user interaction, the command executed as a result of the detected user interaction. ### Claim 12 A system comprising: a tracking module, stored in memory, that tracks a gaze of a user as the user interacts with a user interface; a determining module, stored in memory, that determines, based on tracking of the gaze of the user, that a detected user interaction with the user interface represents a false positive input inference by the user interface; an executing module, stored in memory, that executes at least one remedial action based on determining that the detected user interaction represents the false positive input inference by the user interface; and at least one physical processor that executes the tracking module, the determining module, and the executing module. ### Claim 13 (depends on 12) The system of claim 12, wherein the tracking module tracks the gaze of the user by extracting at least one gaze feature from the gaze of the user as the user interacts with the user interface. ### Claim 14 (depends on 13) The system of claim 13, wherein the at least one gaze feature comprises at least one of: a fixation duration; an angular displacement between an initial fixation centroid and a subsequent fixation centroid; an angular displacement between an initial saccade centroid and a subsequent saccade centroid; an angular displacement between an initial saccade landing point and a subsequent saccade landing point; an amplitude of a saccade; a duration of a saccade; a fixation probability; a saccade probability; a gaze velocity; or a gaze dispersion. ### Claim 15 (depends on 12) The system of claim 12, wherein the determining module determines, based on tracking of the gaze of the user, that the detected user interaction with the user interface represents the false positive input inference by the user interface by: training, using gaze features of the user, a machine learning model to discriminate between true positive events and false positive events; and analyzing the tracked gaze of the user using the trained machine learning model. ### Claim 16 (depends on 12) The system of claim 12, wherein the determining module determines, based on tracking of the gaze of the user, that the detected user interaction with the user interface represents the false positive input inference by the user interface by: training, using gaze features of a group of users, a machine learning model to discriminate between true positive events and false positive events; and analyzing the tracked gaze of the user using the trained machine learning model. ### Claim 17 (depends on 12) The system of claim 12, wherein: the executing module executes the at least one remedial action by receiving, via the user interface, user input associated with the false positive input inference; and the determining module further determines, based on additional tracking of the gaze of the user and the user input associated with the false positive input inference, that an additional detected user interaction with the user interface represents an additional false positive input inference by the user interface. ### Claim 18 (depends on 12) The system of claim 12, wherein the executing module executes the at least one remedial action by: determining that the detected user interaction with the user interface caused a change in an application state of an application associated with the user interface; and automatically undoing the change in the application state. ### Claim 19 A non-transitory computer-readable medium comprising computer-readable instructions that, when executed by at least one processor of a computing system, cause the computing system to: track a gaze of a user as the user interacts with a user interface; determine, based on tracking of the gaze of the user, that a detected user interaction with the user interface represents a false positive input inference by the user interface; and execute at least one remedial action based on determining that the detected user interaction represents the false positive input inference by the user interface. ### Claim 20 (depends on 19) The non-transitory computer-readable medium of claim 19, wherein the computer-readable instructions, when executed by the at least one processor of the computing system, cause the computing system to track the gaze of the user by extracting at least one gaze feature from the gaze of the user as the user interacts with the user interface.