# US 2019/0377477 A1 · Estimations within displays

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0377477 A1
- Document ID: US20190377477A1
- Status: Published application
- Kind: A1
- Application number: 16/476,023
- Filed: 2017-03-21
- Published: 2019-12-12
- Earliest claimed: 2017-03-21
- Assignees: Hewlett-Packard Development Company, L.P.
- Applicants: Hewlett-Packard Development Company, L.P.
- Inventors: Brandon James Lee Haist; Alexander Wayne Clark; Ricky Thomas Burks
- Web page: https://patentgrep.com/patent/US20190377477A1

## CPC

- A61B 5/1114: {Tracking parts of the body}
- A61B 5/6803: {Head-worn items, e.g. helmets, masks, headphones or goggles}
- G02B 27/0093: {with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking}
- G02B 27/017: {Head mounted}
- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06F 3/015: {Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/023: Arrangements for converting discrete items of information into a coded form, e.g. arrangements for interpreting keyboard generated codes as alphanumeric codes, operand codes or instruction codes
- G06F 3/03543: {Mice or pucks (G06F3/03541 takes precedence)}
- G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence)
- A61B 3/113: for determining or recording eye movement
- A61B 5/16: Devices for psychotechnics (using teaching or educational appliances G09B1/00 - G09B7/00); Testing reaction times {; Devices for evaluating the psychological state}
- G02B 2027/0138: {comprising image capture systems, e.g. camera}
- G02B 2027/0187: {slaved to motion of at least a part of the body of the user, e.g. head, eye}
- G06F 2203/011: Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
- G06F 2218/08: Feature extraction
- G06F 2218/12: Classification; Matching
- G06V 10/803: {of input or preprocessed data}
- G06V 20/20: in augmented reality scenes
- G06V 40/193: {Preprocessing; Feature extraction}
- G06V 40/20: Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16)
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Example implementations relate to virtual reality/augmented reality signals. For example, a non-transitory computer readable medium storing instructions executable by a processing resource to receive eye-tracking signals from a camera mounted on a display. The instructions can cause the processing resource to estimate a location on a display using the eye-tracking signals. The instructions can cause the processing resource to receive electroencephalographic (EEG) signals. The instructions can cause the processing resource to confirm the estimated location based on the eye-tracking signals and the EEG signals.

## Claims

### Claim 1

A non-transitory computer readable medium storing instructions executable by a processing resource to:
receive eye-tracking signals from a camera mounted on a display;
estimate a location on a display screen using the eye-tracking signals;
receive electroencephalographic (EEG) signals; and
confirm the estimated location based on the eye-tracking signals and the EEG signals.

### Claim 2 (depends on 1)

The medium of claim 1, further comprising instructions executable by the processing resource to estimate the location based on analysis of previously received eye-tracking signals.

### Claim 3 (depends on 2)

The medium of claim 2, further comprising instructions executable by the processing resource to estimate the location based on:
the analysis of the previously received eye-tracking signals; and
an estimation success of the analysis of the previously received eye-tracking signals.

### Claim 4 (depends on 1)

The medium of claim 1, wherein the eye-tracking signals comprises a movement of pupils of a user captured by the camera.

### Claim 5 (depends on 1)

The medium of claim 1, further comprising instructions executable by the processing resource to confirm the estimated location in response to detection of an excitation response in the EEG signals that corresponds to the eye-tracking signals.

### Claim 6

A method, comprising:
tracking eye movement of a user using camera mounted on a display;
analyzing the eye movement to estimate a location to be selected on a display screen displayed to the user;
recording electroencephalographic (EEG) signals of the user using EEG probes;
correlating the tracked eye movement and the recorded EEG signals to confirm the estimated location; and
selecting the estimated location on the display screen based on the correlation.

### Claim 7 (depends on 6)

The method of claim 6, wherein selecting the estimated location comprises selecting a menu item in a virtual reality system.

### Claim 8 (depends on 6)

The method of claim 6, comprising detecting an input from a user indicating the location to be selected, wherein the input comprises a keyboard input, a mouse input, a handset input, a controller input, a gesture, and combinations thereof.

### Claim 9 (depends on 6)

The method of claim 6, comprising:
detecting a gesture from the user indicating the location to be selected; and
correlating the gesture with the tracked eye movement and the recorded EEG signals to confirm the estimated location.

### Claim 10 (depends on 6)

The method of claim 6, wherein, when the tracked eye movement and the recorded EEG signals do not correlate, indicating that the estimated location is not confirmed.

### Claim 11

An apparatus, comprising:
an infrared (IR) camera mounted on a display;
electroencephalographic (EEG) sensory probes;
a processing resource; and
a memory resource storing machine readable instructions to cause the processing resource to:
track eye movements using the IR camera;
estimate an item to select on a display screen of the display based on the tracked eye movements;
detect EEG signals using the EEG sensory probes; and
confirm the estimated item to select based on the detected EEG signals.

### Claim 12 (depends on 11)

The apparatus of claim 11, wherein the IR camera further comprises two IR cameras positioned in front of and facing toward eyes of a user of the apparatus.

### Claim 13 (depends on 11)

The apparatus of claim 11, further comprising a camera positioned to detect gesture movement of a user.

### Claim 14 (depends on 13)

The apparatus of claim 13, further comprising instructions executable by the processing resource to confirm the estimated item based on the detected gestures.

### Claim 15 (depends on 11)

The apparatus of claim 11, wherein the IR camera are positioned a particular distance from eyes of a user to monitor a gaze of the eyes.
