# US 2019/0171283 A1 · Modifying a computer-based interaction based on eye gaze

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0171283 A1
- Document ID: US20190171283A1
- Status: Published application
- Kind: A1
- Application number: 15/830,055
- Filed: 2017-12-04
- Published: 2019-06-06
- Earliest claimed: 2017-12-04
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: KUNTAL DEY; ANIL U. JOSHI; PUTHUKODE G. RAMACHANDRAN; WILLIE L. SCOTT, II
- Web page: https://patentgrep.com/patent/US20190171283A1

## CPC

- G06V 40/193: {Preprocessing; Feature extraction}
- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06F 3/0481: based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
- G06F 3/04895: Guidance during keyboard input operation, e.g. prompting
- G06F 18/23213: with fixed number of clusters, e.g. K-means clustering
- G06N 20/00: Machine learning
- G06F 9/453: {Help systems}
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A method modifies a computer-based interaction based on gaze data. One or more processors collect eye gaze data points to create an eye gaze corpus of information, where the eye gaze data points describe an eye gaze of viewers of a first set of at least one user interface. The processor(s) generate a plurality of clusters of viewers, and determine a target action performance for each of the plurality of clusters. The processor(s) collect, from a device having eye tracking technology, real time eye gaze data from a plurality of current users who are viewing a second set of at least one user interface, and segment the plurality of current users. The processor(s) then modify a computer-based interaction for at least one segment in order to maximize target action performance of the second set of at least one user interface.

## Claims

### Claim 1

A method comprising:
collecting, by one or more processors, eye gaze data points to create an eye gaze corpus of information, wherein the eye gaze data points describe an eye gaze of viewers of a first set of at least one user interface;
applying, by one or more processors, an unsupervised machine learning algorithm to the eye gaze corpus to generate a plurality of clusters based upon the eye gaze data points;
determining, by one or more processors, a target action performance of the first set of at least one user interface for each of the plurality of clusters;
collecting, by one or more processors and from a device having eye tracking technology, real time eye gaze data from a plurality of current users who are viewing a second set of at least one user interface;
segmenting, by one or more processors, the plurality of current users based upon the plurality of clusters by analyzing patterns among the real-time eye gaze data and by analyzing scenario data pertaining to the target action performance; and
responsive to the segmenting of the plurality of current users, modifying, by one or more processors, a computer-based interaction for at least one segment of the plurality of current users in order to maximize target action performance of the second set of at least one user interface.

### Claim 2 (depends on 1)

The method of claim 1, wherein the unsupervised machine learning algorithm is a k-means clustering algorithm.

### Claim 3 (depends on 1)

The method of claim 1, wherein the scenario data pertaining to the target action performance includes historical data on failed scenarios with respect to completion of at least one target action.

### Claim 4 (depends on 1)

The method of claim 1, wherein the scenario data pertaining to the target action performance includes historical data on successful scenarios with respect to completion of at least one target action.

### Claim 5 (depends on 1)

The method of claim 1, wherein the scenario data includes historical data that is based upon profile data associated with at least one of the plurality of current users.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
categorizing, by one or more processors, the plurality of clusters based upon a predefined significance of one or more aspects of the segmenting.

### Claim 7 (depends on 1)

The method of claim 1, wherein modifying the computer-based interaction with at least one segment of the plurality of current users comprises modifying at least one aspect of a graphical user interface of a software program.

### Claim 8 (depends on 7)

The method of claim 7, wherein modifying at least one aspect of the graphical user interface comprises displaying promotional content via the graphical user interface.

### Claim 9 (depends on 7)

The method of claim 7, wherein modifying at least one aspect of the graphical user interface comprises displaying additional clarification content related to the software program via the graphical user interface.

### Claim 10 (depends on 1)

The method of claim 1, wherein modifying the computer-based interaction with at least one segment of the plurality of current users comprises presenting a conversational interface on a graphical user interface.

### Claim 11 (depends on 1)

The method of claim 1, wherein the first set of at least one user interface and the second set of at least one user interface are a same set of at least one user interface.

### Claim 12 (depends on 1)

The method of claim 1, wherein the first set of at least one user interface and the second set of at least one user interface are different sets of at least one user interface.

### Claim 13 (depends on 1)

The method of claim 1, wherein the eye gaze of the viewers of the first set of at least one user interface and the real-time eye gaze data from the plurality of current users who are viewing the second set of at least one user interface are based on a pathological nystagmus of the viewers of the first set of at least one user interface and the plurality of current users who are viewing the second set of at least one user interface.

### Claim 14 (depends on 1)

The method of claim 1, wherein the eye gaze of the viewers of the first set of at least one user interface and the real-time eye gaze data from the plurality of current users who are viewing the second set of at least one user interface are based on a position on a graphical user interface at which the viewers of the first set of at least one user interface and the plurality of current users who are viewing the second set of at least one user interface are visually focused.

### Claim 15

A computer program product for modifying a computer-based interaction based on historical eye gaze data, the computer program product comprising a non-transitory computer readable storage device having program instructions embodied therewith, the program instructions readable and executable by a computer to perform a method comprising:
collecting eye gaze data points to create an eye gaze corpus of information, wherein the eye gaze data points describe an eye gaze of viewers of a first set of at least one user interface;
applying an unsupervised machine learning algorithm to the eye gaze corpus to generate a plurality of clusters based upon the eye gaze data points;
determining a target action performance of the first set of at least one user interface for each of the plurality of clusters;
collecting, from a device having eye tracking technology, real time eye gaze data from a plurality of current users who are viewing a second set of at least one user interface;
segmenting the plurality of current users based upon the plurality of clusters by analyzing patterns among the real-time eye gaze data and by analyzing scenario data pertaining to the target action performance; and
responsive to the segmenting of the plurality of current users, modifying a computer-based interaction for at least one segment of the plurality of current users in order to maximize target action performance of the second set of at least one user interface.

### Claim 16 (depends on 15)

The computer program product of claim 15, wherein modifying the computer-based interaction with at least one of the plurality of current users comprises presenting a conversational interface on a graphical user interface.

### Claim 17 (depends on 15)

The computer program product of claim 15, wherein the program instructions are provided as a service in a cloud environment.

### Claim 18

A computer system comprising one or more processors, one or more computer readable memories, one or more computer readable storage mediums, and program instructions stored on at least one of the one or more storage mediums for execution by at least one of the one or more processors via at least one of the one or more memories, the stored program instructions comprising:
program instructions to collect eye gaze data points to create an eye gaze corpus of information, wherein the eye gaze data points describe an eye gaze of viewers of a first set of at least one user interface;
program instructions to apply an unsupervised machine learning algorithm to the eye gaze corpus to generate a plurality of clusters based upon the eye gaze data points;
program instructions to determine a target action performance of the first set of at least one user interface for each of the plurality of clusters;
program instructions to collect, from a device having eye tracking technology, real time eye gaze data from a plurality of current users who are viewing a second set of at least one user interface;
program instructions to segment the plurality of current users based upon the plurality of clusters by analyzing patterns among the real-time eye gaze data and by analyzing scenario data pertaining to the target action performance; and
program instructions to, responsive to segmenting the plurality of current users, modify a computer-based interaction for at least one segment of the plurality of current users in order to maximize target action performance of the second set of at least one user interface.

### Claim 19 (depends on 18)

The computer system of claim 18, wherein modifying the computer-based interaction with at least one of the plurality of current users comprises presenting a conversational interface on a graphical user interface.

### Claim 20 (depends on 18)

The computer system of claim 18, wherein the program instructions are provided as a service in a cloud environment.
