Back to results
US 11,231,777 B2Granted
Method for controlling device on the basis of eyeball motion, and device therefor
AssigneeSamsung Electronics Co., Ltd.
InventorsJun-seok Lee, Kyung-hwa Yu, Sang-won Leigh, Jin-yong Chung, Sung-goo Cho
Life of this patent
Estimating the term…
Abstract
A method of controlling a device is provided. The method includes receiving eye movement information of a user. A gazed position within a display area of the device is identified based on the eye movement information of the user. A position of a cursor corresponding to the gazed position is corrected based on the eye movement information. A control command corresponding to an area within the display area of the device is generated, based on the gazed position and the eye movement information. An operation of the device is controlled based on the control command.
Claims18
An electronic device comprising:
- a display;
- a memory storing instructions;
- and
- at least one processor configured to execute the stored instructions to at least:
- obtain eye action related information of a user by using a camera device coupled to the electronic device, the eye action related information including eye movement information and eyelid movement information of the user;
- divide the display of the electronic device, except an area in which a content is displayed, into a plurality of areas;
- identify a gazed position within a display area of the electronic device based on the eye movement information of the user;
- correct a position of a cursor corresponding to the gazed position to match the gazed position based on the eye movement information;
- identify an area of the plurality of areas, based on the corrected position of the cursor;
- based at least on the eye movement information and the eyelid movement information of the user meeting a predetermined criteria, initiate a minute input adjustment function for minutely moving the cursor on the display of the electronic device, and obtain first eye gazed position related information of the user and identify a first eye gazed position on the display that is gazed at by the user based on the first eye gazed position related information;
- based at least on the initiating the minute input adjustment function, display, on the display of the electronic device, a rectangular frame and a cursor corresponding to the first eye gazed position on the display, the cursor located at a center within the rectangular frame;
- obtain second eye gazed position related information of the user by using the camera device coupled to the electronic device while the rectangular frame and the cursor are displayed, and identify a second eye gazed position on the display that is gazed at by the user based on the second eye gazed position related information;
- determine a relationship between the second eye gazed position on the display and a position on the rectangular frame;
- based at least on determining the relationship, identify a direction for minutely moving the cursor while the rectangular frame and the cursor are displayed;
- and
- minutely move, on the display of the electronic device, the cursor located within the rectangular frame in the identified direction.
13 more elements
A method performed by an electronic device, the method comprising:
- obtaining eye action related information of a user by using a camera device coupled to the electronic device, the eye action related information including eye movement information and eyelid movement information of the user;
- dividing a display of the electronic device, except an area in which a content is displayed, into a plurality of areas;
- identifying a gazed position within a display area of the electronic device based on the eye movement information of the user;
- correcting a position of a cursor corresponding to the gazed position to match the gazed position based on the eye movement information;
- identifying an area of the plurality of areas, based on the corrected position of the cursor;
- based at least on the eye movement information and the eyelid movement information of the user meeting a predetermined criteria, initiating a minute input adjustment function for minutely moving the cursor on the display of the electronic device, and obtaining a first eye gazed position related information of the user and identifying a first eye gazed position on the display that is gazed at by the user based on the first eye gazed position related information;
- based at least on the initiating the minute input adjustment function, displaying, on the display of the electronic device, a rectangular frame and a cursor corresponding to the first eye gazed position on the display, the cursor located at a center within the rectangular frame;
- obtaining second eye gazed position related information of the user by using the camera device coupled to the electronic device while the rectangular frame and the cursor are displayed, and identifying a second eye gazed position on the display that is gazed at by the user based on the second eye gazed position related information;
- determining a relationship between the second eye gazed position and a position on the rectangular frame;
- based at least on determining the relationship, identifying a direction for minutely moving the cursor while the rectangular frame and the cursor are displayed;
- and
- minutely moving, on the display of the electronic device, the cursor located within the rectangular frame in the identified direction.
9 more elements
A non-transitory computer readable recording medium storing instructions executable by at least one processor of an electronic device to cause the at least one processor to at least:
- obtain eye action related information of a user by using a camera device coupled to the electronic device, the eye action related information including eye movement information and eyelid movement information of the user;
- divide a display of the electronic device, except an area in which a content is displayed, into a plurality of areas;
- identify a gazed position within a display area of the electronic device based on the eye movement information of the user;
- correct a position of a cursor corresponding to the gazed position to match the gazed position based on the eye movement information;
- identify an area of the plurality of areas, based on the corrected position of the cursor;
- based at least on the eye movement information and the eyelid movement information of the user meeting a predetermined criteria, initiate a minute input adjustment function for minutely moving the cursor on the display of the electronic device, and obtain first eye gazed position related information of the user and identify a first eye gazed position on the display that is gazed at by the user based on the first eye gazed position related information;
- based at least on the initiating the minute input adjustment function, display, on the display of the electronic device, a rectangular frame and a cursor corresponding to the first eye gazed position on the display, the cursor located at a center within the rectangular frame;
- obtain second eye gazed position related information of the user by using the camera device coupled to the electronic device while the rectangular frame and the cursor are displayed, and identify a second eye gazed position on the display that is gazed at by the user based on the second eye gazed position related information;
- determine a relationship between the second eye gazed position and a position on the rectangular frame;
- based at least on determining the relationship, identify a direction for minutely moving the cursor while the rectangular frame and the cursor are displayed;
- and
- minutely move, on the display of the electronic device, the cursor located within the rectangular frame in the identified direction.
9 more elements
Dependencies are inferred from the claim text.