# US 10,481,685 B2 · Method for controlling device on the basis of eyeball motion, and device therefor

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,481,685 B2
- Document ID: US10481685B2
- Status: Granted
- Kind: B2
- Application number: 15/864,745
- Filed: 2018-01-08
- Published: 2018-05-10
- Granted: 2019-11-19
- Earliest claimed: 2012-03-08
- Assignees: Samsung Electronics Co., Ltd.
- Applicants: Samsung Electronics Co., Ltd.
- Inventors: Jun-seok Lee; Kyung-hwa Yu; Sang-won Leigh; Jin-yong Chung; Sung-goo Cho
- Web page: https://patentgrep.com/patent/US10481685B2

## CPC

- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/033: Pointing devices displaced or positioned by the user {, e.g. mice, trackballs, pens or joysticks}; Accessories therefor (digitisers characterised by the transducing means G06F3/041)
- G06F 3/0482: Interaction with lists of selectable items, e.g. menus
- G06F 3/0484: for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F 3/0485: Scrolling or panning
- G06V 40/18: Eye characteristics, e.g. of the iris
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A method of controlling an operation of a display device using eye movements and a device for performing the method are provided. The method includes receiving eye movement information of a user; receiving blinking information of the user; generating a control command corresponding to the eye movement information and the blinking information; and controlling an operation of the display device based on the generated control command.

## Claims

### Claim 1

A method of controlling a device comprising:
receiving eye movement information of a user;
dividing a display of the display device except an area in which contents are displayed into a plurality of areas;
identifying a gazed position within a display area of the device based on the eye movement information of the user;
identifying an area of the plurality of areas, based on the gazed position;
determining whether a control command is associated with the area that is identified; and
controlling an operation of the device based on the control command in response to determining that a control command is associated with the area that is identified.

### Claim 2 (depends on 1)

The method of claim 1,
wherein the method comprises generating the control command based on an area gazed at by the user from among the divided areas and a movement direction of the gaze of the user from the area gazed at by the user.

### Claim 3 (depends on 1)

The method of claim 1, wherein the determining the control command comprises identifying the control command based on gazed position from the display to an area outside of the display.

### Claim 4 (depends on 1)

The method of claim 1, further comprising:
displaying a control command selection window comprising a list of control commands on a display of the device;
sensing a selection of a control command from the control command list included in the control command selection window based on a user input via the displayed control command selection window; and
moving a cursor on the display of the device to a predetermined object based on the eye movement information of the user; and
wherein the determining comprises identifying the control command corresponding to the selected control command with respect to the object in response to an eyelid movement by the user.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
fixing a position of a cursor corresponding to the gazed position; and
moving a position of the cursor to correspond to a moved the gazed position, in response to the eye movement information of the user.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
displaying a previously set area around a cursor based on a user input; and
moving a cursor based on the gazed position with respect to the displayed area.

### Claim 7 (depends on 1)

The method of claim 1, further comprising:
moving a cursor to an object in response to the cursor being moved based on the eye movement information to an area that is located within a previously set range of the object on the display of the device.

### Claim 8 (depends on 1)

A non-transitory computer readable medium comprising a computer program that is executable by a computer to perform the method of claim 1.

### Claim 9 (depends on 1)

The method of claim 1, wherein the gazed position and an eyelid movement information are determined based on the eye movement information, and
the control command is generated based on the position of the gaze, the movement direction of the gaze, and the eyelid movement information.

### Claim 10

A device for operating based on eye movements of a user, the device comprising:
a memory storing instructions; and
a processor configured to execute the instructions to implement:
a screen divider configured to divide the display except an area in which contents are displayed into a plurality of areas;
an eye movement information receiver configured to receive information about eye movements of a user, identify a gazed position within a display area of the device based on the eye movement information of the user, and
a control command generator configured to identify an area of the plurality of areas, based on the gazed position, determine whether a control command is associated with the area that is identified, and control an operation of the device based on the control command in response to determining that a control command is associated with the area that is identified.

### Claim 11 (depends on 10)

The device of claim 10, wherein the processor is configured to identify the control command based on a movement of a gaze of the user as the gazed position moves from the display to an area outside of the display.

### Claim 12 (depends on 10)

The device of claim 10, further comprising:
a display configured to display a control command selection window including a list of control commands corresponding to an eyelid movement of the user; and
a sensor configured to sense a user input via the displayed control command selection window and the eyelid movement of the user to select a control command from the list,
wherein the processor is configured to identify the control command corresponding to the sensed selected control command with respect to an object placed at a position of a cursor.

### Claim 13 (depends on 10)

The device of claim 10, wherein the processor configured to correct a position of a cursor on a display corresponding to a gaze of the user and match a position of a moved gaze of the user with the position of the cursor, in response to a movement of the gazed position.

### Claim 14 (depends on 10)

The device of claim 10, further comprising:
a display configured to display a previously set area around a cursor based on a user input; and
an input adjuster,
wherein the processor is configured to move the cursor with respect to the displayed area, based on a direction of a position at which the gazed position is located.

### Claim 15 (depends on 10)

The device of claim 10, further comprising:
a display configured to move a cursor on the display to an object on the display, in response to the cursor being moved based on the eye movement information to an area that is located within a previously set range of the object on the display of the device.

### Claim 16 (depends on 10)

The device of claim 10, wherein the processor is configured to determine the gazed position and an eyelid movement information based on the eye movement information, and generate the control command based on the position of the gaze, the movement direction of the gaze, and the eyelid movement information.

### Claim 17

A non-transitory computer-readable medium having stored thereon instructions executable by a device to cause the device to:
receive eye movement information of a user;
divide a display of the device except an area in which contents are displayed into a plurality of areas;
identify a gazed position within a display area of the device based on the eye movement information of the user;
identify an area of the plurality of areas, based on the gazed position;
determine whether a control command is associated with the area that is identified; and
control an operation of the device based on the control command in response to determining that a control command is associated with the area that is identified.