# US 10,481,683 B2 · Eye-brain interface (EBI) system and method for controlling same

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,481,683 B2
- Document ID: US10481683B2
- Status: Granted
- Kind: B2
- Application number: 15/740,298
- Filed: 2015-12-17
- Published: 2018-07-12
- Granted: 2019-11-19
- Earliest claimed: 2015-12-17
- Assignees: Looxid Labs Inc.
- Applicants: Looxid Labs Inc.
- Inventors: Yongwook Chae
- Web page: https://patentgrep.com/patent/US10481683B2

## CPC

- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- A61B 5/16: Devices for psychotechnics (using teaching or educational appliances G09B1/00 - G09B7/00); Testing reaction times {; Devices for evaluating the psychological state}
- G06F 1/163: {Wearable computers, e.g. on a belt}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/015: {Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence)
- G06F 3/0487: using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
- A61B 5/0024: {for multiple sensor units attached to the patient, e.g. using a body or personal area network}
- G06F 2203/0381: Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)

## Abstract

Methods and systems for calibrating an eye-brain interface (EBI) system controlled on the basis of eye movements and brain waves according to one embodiment of the present invention are disclosed. A method includes: providing an eye-brain calibration (EBC) interface for calibrating eye movements and brain waves simultaneously, wherein the EBC interface comprises a visual object and instructs a user to gaze into the visual object in a particular cognitive state; acquiring eye movements and brain waves of the user for the visual object included in the EBC interface; mapping the visual object and eye movements of the user; and mapping the particular cognitive state instructed to the user and brain waves of the user.

## Claims

### Claim 1

A method for calibrating an eye-brain interface (EBI) system that is controlled based on eye tracking and brain waves, the method comprising:
providing an eye-brain calibration (EBC) interface for calibrating gaze position and brain waves together, wherein the EBC interface comprises a visual object and instructs a user to gaze the visual object in a particular cognitive state;
obtaining the user's gaze position and brain wave on the visual object included in the EBC interface;
mapping the visual object with the user's gaze position; and
mapping the particular cognitive state with the user's brain wave,
wherein the EBC interface sequentially and/or alternately provides a first visual object instructing the user to view the first visual object in a first cognitive state and a second visual object instructing the user to view the second visual object in a second cognitive state.

### Claim 2 (depends on 1)

The method of claim 1, wherein the mapping the user's gaze position comprises mapping coordinates of the visual object on a screen with coordinates of the user's gaze position.

### Claim 3 (depends on 1)

The method of claim 1, wherein the first cognitive state is a cognitive state of attention and/or selection, and the second cognitive state is a cognitive state of rest and/or search.

### Claim 4 (depends on 1)

The method of claim 1, wherein the mapping the user's brain wave comprises:
obtaining first raw data on a brain wave in the first cognitive state, and second raw data on a brain wave in the second cognitive state;
converting frequencies of the first raw data and the second raw data; and
setting a criterion for classifying the first and second cognitive states based on frequency characteristics of the frequency-converted first raw data and second raw data.

### Claim 5 (depends on 4)

The method of claim 4, wherein the setting the criterion comprises:
extracting a frequency amplitude for each of frequency bands in a predetermined range from the frequency-converted first raw data and second raw data;
obtaining a Fisher's ratio for each of the frequency bands using the extracted frequency amplitude;
selecting a first frequency band having a highest Fisher's ratio and a second frequency band having a second highest Fisher's ratio; and
setting the first and second frequency bands as criteria for classifying the first and second cognitive states.

### Claim 6 (depends on 5)

The method of claim 5, wherein the Fisher's ratio is calculated based on an average and a variance of the frequency amplitudes in the frequency-converted first raw data and an average and a variance of the frequency amplitudes in the frequency-converted second raw data.

### Claim 7 (depends on 5)

The method of claim 5, wherein the frequency band in the predetermined range corresponds to δ-wave band, θ-wave band, α-wave band or β-wave band of a brain wave.

### Claim 8 (depends on 1)

The method of claim 1, wherein the EBC interface adjusts a flickering frequency of the visual object to thereby induce the user's brain wave to a particular frequency band.

### Claim 9 (depends on 8)

The method of claim 8, wherein the EBC interface adjusts the flickering frequency of the visual object to approximately 8 to 13 Hz to induce the user's brain wave to an alpha wave range, and adjusts the flickering frequency of the visual object to approximately 13 to 30 Hz to induce the user's brain wave to a beta wave range.

### Claim 10 (depends on 1)

The method of claim 1, further comprising:
obtaining an iris image from the user's eye; and coding the iris image.

### Claim 11 (depends on 10)

The method of claim 10, wherein the coding the iris image comprises:
dividing the obtained iris image into a plurality of images;
arranging the plurality of images in one direction; and
converting the images arranged in the one direction into a single two-dimensional image.

### Claim 12

A slave device for measuring gaze position and brain waves, comprising:
an eye tracker configured to track an eye of a user;
a brain wave sensor configured to sense brain wave of the user;
a communication interface configured to conduct communications with a host device; and
a processor configured to control the eye tracker, the brain wave sensor and the communication interface,
wherein the host device provides an eye-brain calibration (EBC) interface for simultaneously calibrating gaze position and brain waves, the EBC interface comprising a visual object and instructing the user to gaze at the visual object in a particular cognitive state, and
wherein the processor is configured to, upon receiving a calibration start signal from the host device, obtain the gaze position and brain wave of the user together and transmit the gaze position and the brain wave of the user to the host device,
wherein the EBC interface is configured to sequentially and/or alternately provide a first visual object instructing the user to view the first visual object in a first cognitive state and a second visual object instructing the user to view the second visual object in a second cognitive state.

### Claim 13

A host device controlled based on eye tracking and brain waves, comprising:
a display configured to display an image;
a communication interface configured to conduct communications with a slave device; and
a processor configured to control the display and the communication interface,
wherein the processor is configured to provide an eye-brain calibration (EBC) interface for simultaneously calibrating gaze position and brain waves, the EBC interface comprising a visual object and instructing a user to gaze at the visual object in a particular cognitive state,
wherein the processor is configured to request and receive the gaze position and brain waves of the user from the slave device, map the visual object with the gaze position of the user, and map the brain wave of the user with the particular cognitive state,
wherein the EBC interface sequentially and/or alternately provides a first visual object instructing the user to view the first visual object in a first cognitive state and a second visual object instructing the user to view the second visual object in a second cognitive state.

### Claim 14 (depends on 13)

The host device of claim 13, wherein the processor maps coordinates of the visual object on a screen with coordinates of the gaze position of the user when it maps the gaze position of the user.

### Claim 15 (depends on 13)

The host device of claim 13, wherein the first cognitive state is a cognitive state of attention or selection, and the second cognitive state is a cognitive state of rest or search.

### Claim 16 (depends on 13)

The host device of claim 13, wherein the processor, when it maps the brain wave of the user,
obtains first raw data on a brain wave in the first cognitive state, and second raw data on a brain wave in the second cognitive state,
converts frequencies of the first raw data and the second raw data,
extracts a frequency amplitude for each of frequency bands in a predetermined range from the frequency-converted first raw data and second raw data,
obtains a Fisher's ratio for each of the frequency bands using the extracted frequency amplitude,
selects a first frequency band having a highest Fisher's ratio and a second frequency band having a second highest Fisher's ratio, and
sets the first and second frequency bands as criteria for classifying the first and second cognitive states.

### Claim 17 (depends on 16)

The host device of claim 16, wherein the processor is configured to obtain the brain waves of the user in real-time and classify the brain waves of the user obtained in real-time according to the criteria in real-time.

### Claim 18 (depends on 13)

The host device of claim 13, wherein the EBC interface adjusts a flickering frequency of the visual object to thereby induce the brain wave of the user to a particular frequency band.