# US 10,372,206 B2 · Eye-controlled apparatus, eye-controlled method and eye-controlled system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,372,206 B2 - Document ID: US10372206B2 - Status: Granted - Kind: B2 - Application number: 15/511,861 - Filed: 2016-04-14 - Published: 2017-10-12 - Granted: 2019-08-06 - Earliest claimed: 2015-05-27 - Assignees: Boe Technology Group Co., Ltd. - Applicants: BOE Technology Group Co., Ltd. - Inventors: Wenbo Li; Tian Yang - Web page: https://patentgrep.com/patent/US10372206B2 ## CPC - G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)} - G06F 3/14: Digital output to display device {; Cooperation and interconnection of the display device with other functional units} - G09G 2320/0261: in the context of movement of objects on the screen or movement of the observer relative to the screen - G09G 2354/00: Aspects of interface with display user - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00) ## Abstract The present disclosure discloses an eye-controlled apparatus, an eye-controlled method and an eye-controlled system. The eye-controlled apparatus includes a fixation point acquisition unit, a human eye action detection unit and a control signal generation unit. The fixation point acquisition unit is configured to acquire position information about the fixation point of human eyes on a device to be operated. The human eye action detection unit is configured to detect whether the human eyes take a pre-set action, and control the fixation point acquisition unit to transmit current position information about the fixation point of the human eyes on the device to be operated to the control signal generation unit when detecting the preset action taken by the human eyes. The control signal generation unit is configured to generate, based on a pre-stored position control correspondence table corresponding to the device to be operated, a control signal corresponding to the current position information about the fixation point of the human eyes on the device to be operated, and transmit the control signal to the device to be operated so as to control the device to be operated to execute a corresponding operation. The eye-controlled apparatus, the eye-controlled method and the eye-controlled system may effectively control the device to be operated using human eyes. ## Claims ### Claim 1 An eye-controlled apparatus, comprising: A fixation point acquisition unit configured to acquire position information about a fixation point of human eyes on a device to be operated; a human eye action detection unit configured to detect whether the human eyes take a pre-set action, and control the fixation point acquisition unit to output current position information about the fixation point of the human eyes on the device to be operated when detecting the preset action taken by the human eyes; and a control signal generation unit configured to generate, based on a pre-stored position control correspondence table corresponding to the device to be operated, a control signal corresponding to the current position information about the fixation point of the human eyes on the device to be operated, and transmit the control signal to the device to be operated so as to control the device to execute a corresponding operation, wherein position information on the device to be operated and the control signals corresponding to position information are stored in the position control correspondence table, respectively, wherein the fixation point acquisition unit comprises: an infrared emission module provided on the device to be operated and adapted to emit infrared light to the human eyes and form light reflection points in two pupils of the human eyes; a first human eye image acquisition module provided on the device to be operated and adapted to acquire an image of the human eyes; an image processing module configured to establish an image coordinate system based on the image of the human eyes acquired by the first human eye image acquisition module, and process the image of the human eyes to obtain position coordinates of centers of the two pupils and of the light reflection points in the image coordinate system; and a calculation module configured to obtain the position information about the fixation point of the human eyes on the device to be operated by a cross ratio invariance algorithm based on the position coordinates of centers of the two pupils and of the light reflection points in the image coordinate system obtained by the image processing module. ### Claim 2 (depends on 1) The eye-controlled apparatus according to claim 1, wherein the infrared emission module comprises four infrared emission sources provided at four corners of the device to be operated, respectively. ### Claim 3 (depends on 1) The eye-controlled apparatus according to claim 1, wherein the fixation point acquisition unit further comprises a correction module configured to correct the position information about the fixation point of the human eyes on the device to be operated according to a linear scaling algorithm. ### Claim 4 (depends on 1) The eye-controlled apparatus according to claim 1, wherein the image processing module and the calculation module are integrated in the device to be operated. ### Claim 5 An eye-controlled apparatus, comprising: A fixation point acquisition unit configured to acquire position information about a fixation point of human eyes on a device to be operated; a human eye action detection unit configured to detect whether the human eyes take a pre-set action, and control the fixation point acquisition unit to output current position information about the fixation point of the human eyes on the device to be operated when detecting the preset action taken by the human eyes; and a control signal generation unit configured to generate, based on a pre-stored position control correspondence table corresponding to the device to be operated, a control signal corresponding to the current position information about the fixation point of the human eyes on the device to be operated, and transmit the control signal to the device to be operated so as to control the device to execute a corresponding operation, wherein position information on the device to be operated and the control signals corresponding to position information are stored in the position control correspondence table, respectively, wherein the fixation point acquisition unit comprises: spectacles; a scene acquisition module provided on the spectacles and adapted to acquire a scene image seen by the human eyes through the spectacles, the scene image containing an image of the device to be operated; a second human eye image acquisition module provided on the spectacles and adapted to acquire an image of the human eyes, a fixation direction determination module configured to determine a fixation direction of the human eyes based on the image of the human eyes acquired by the second human eye image acquisition module; and a fixation point determination module configured to determine the position information about the fixation point of the human eyes on the device to be operated based on the scene image acquired by the scene acquisition module and the fixation direction of the human eyes determined by the fixation direction determination module. ### Claim 6 (depends on 5) The eye-controlled apparatus according to claim 5, wherein the fixation direction determination module is configured to determine the fixation direction of the human eyes based on pupil positions in the image of the human eyes acquired by the second human eye image acquisition module. ### Claim 7 (depends on 1) The eye-controlled apparatus according to claim 1, wherein the preset action comprises keeping the position of the fixation point of the human eyes on the device to be operated unchanged for 2 to 3 seconds, or, blinking rapidly for 3 to 5 times within a preset time while keeping the position of the fixation point of the human eyes on the device to be operated unchanged. ### Claim 8 (depends on 1) The eye-controlled apparatus according to claim 1, wherein the human eye action detection unit is an apparatus having an imaging function. ### Claim 9 (depends on 1) The eye-controlled apparatus according to claim 1, wherein the human eye action detection unit is configured to acquire a real-time image of the human eyes by the first human eye image acquisition module, and obtain the action of the human eyes based on the real-time image of the human eyes. ### Claim 10 (depends on 1) An eye-controlled system comprising: a plurality of devices to be operated and an eye-controlled apparatus according to claim 1. ### Claim 11 (depends on 10) The eye-controlled system according to claim 10, wherein the fixation point acquisition units in the eye-controlled apparatus have the same number as that of the devices to be operated, and are one-to-one corresponding to the devices to be operated; and wherein each fixation point acquisition units is provided on a respective device to be operated. ### Claim 12 (depends on 11) The eye-controlled system according to claim 11, wherein the human eye action detection units in the eye-controlled apparatus have the same number as that of the devices to be operated, and are one-to-one corresponding to the devices to be operated; and each human eye action detection units is provided on a respective device to be operated. ### Claim 13 (depends on 11) The eye-controlled system according to claim 11, wherein all of the fixation point acquisition units are connected to one single control signal generation unit, the control signal generation unit being adapted to store a plurality of position control correspondence tables corresponding to the devices to be operated, respectively, and transmit control information to each device to be operated. ### Claim 14 (depends on 1) An eye-controlled method using the eye-controlled apparatus according to claim 1, comprising steps of: by a fixation point acquisition unit, acquiring the position information about the fixation point of the human eyes on the device to be operated; by a human eye action detection unit, detecting whether the human eyes take the preset action, and controlling the fixation point acquisition unit to transmit the current position information about the fixation point of the human eyes on the device to be operated to a control signal generation unit when detecting the preset action taken by the human eyes; and by the control signal generation unit, generating the control signal corresponding to the current position information about the fixation point of the human eyes on the device to be operated based on the pre-stored position control correspondence table corresponding to the device to be operated, and transmitting the generated control signal to the device to be operated so as to control the device to execute a corresponding operation, wherein the position information on the device to be operated and the control signals corresponding to the position information, respectively, are stored in the position control correspondence table. ### Claim 15 (depends on 14) The eye-controlled method according to claim 14, further comprising a step of: prior to acquiring the position information about the fixation point of the human eyes on the device to be operated by the fixation point acquisition unit, establishing the position control correspondence table corresponding to the device to be operated. ### Claim 16 (depends on 5) The eye-controlled apparatus according to claim 5, wherein the human eye action detection unit is configured to acquire a real-time image of the human eyes by the second human eye image acquisition module, and obtain the action of the human eyes based on the real-time image of the human eyes. ### Claim 17 (depends on 5) An eye-controlled system comprising: a plurality of devices to be operated and an eye-controlled apparatus according to claim 5. ### Claim 18 (depends on 17) The eye-controlled system according to claim 17, wherein the eye-controlled apparatus comprises one fixation point acquisition unit, and wherein the eye-controlled system further comprises a plurality of activation units one-to-one corresponding to the devices to be operated, each activation units being provided on a respective device to be operated, and configured to activate the eye-controlled apparatus when the fixation point acquisition unit is directed to a device to be operated corresponding to the activation unit and control the control signal generation unit to call the position control correspondence table corresponding to the device to be operated corresponding to the activation unit. ### Claim 19 (depends on 18) The eye-controlled system according to claim 18, wherein the eye-controlled apparatus comprises one human eye action detection unit provided on spectacles of the fixation point acquisition unit.