# US 10,372,204 B2 · Communication and control system and method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,372,204 B2
- Document ID: US10372204B2
- Status: Granted
- Kind: B2
- Application number: 15/027,731
- Filed: 2014-10-30
- Published: 2016-08-25
- Granted: 2019-08-06
- Earliest claimed: 2013-10-30
- Assignees: Technology Against Als
- Applicants: Technology Against Als
- Inventors: Sandeep Tungare; Milind Gupte; Amitabh Patil; Naveen Reddy; Bert Haberland
- Web page: https://patentgrep.com/patent/US10372204B2

## CPC

- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- A61F 4/00: Methods or devices enabling patients or disabled persons to operate an apparatus or a device not forming part of the body
- G02B 27/0093: {with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking}
- G02B 27/01: Head-up displays
- G02B 27/017: {Head mounted}
- G06F 3/015: {Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/0236: {using selection techniques to select from displayed items}
- G06F 3/0237: {using prediction or retrieval techniques}
- G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence)
- G06F 3/0487: using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G09B 21/00: Teaching, or communicating with, the blind, deaf or mute (audible presentation of material to be studied G09B5/04; devices or methods for replacing direct visual or auditory perception by another kind of perception A61F9/08, A61F11/04; audible indication of meter readings or of colour G01D7/12; watches for blind persons G04B25/02; speech analysis, speech recognition G10L; sound-recording or reproducing, per se G11B)
- G02B 2027/0141: {characterised by the informative content of the display}
- G02B 2027/0178: {Eyeglass type (eyeglass details G02C)}
- G02B 2027/0187: {slaved to motion of at least a part of the body of the user, e.g. head, eye}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- A61F: FILTERS IMPLANTABLE INTO BLOOD VESSELS; PROSTHESES; DEVICES PROVIDING PATENCY TO, OR PREVENTING COLLAPSING OF, TUBULAR STRUCTURES OF THE BODY, e.g. STENTS; ORTHOPAEDIC, NURSING OR CONTRACEPTIVE DEVICES; FOMENTATION; TREATMENT OR PROTECTION OF EYES OR EARS; BANDAGES, DRESSINGS OR ABSORBENT PADS; FIRST-AID KITS (dental prosthetics A61C)
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G09B: EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS (devices for psychotechnics or for testing reaction times A61B5/16; games, sports, amusements A63; projectors, projector screens G03B)

## Abstract

An input system includes at least one sensor and a controller communicatively coupled to the at least one sensor. The at least one sensor is configured to identify a position of an eye. The controller is configured to receive a first signal indicative of a first position of an eye from the at least one sensor and map the eye position to a user's selection.

## Claims

### Claim 1

A system, comprising:
a support structure for supporting a plurality of visible indicia such that the plurality of visible indicia is located at a periphery of a user's field of vision, the plurality of visible indicia collectively establishing a physical reference frame having a plurality of colors and configured to provide a first feedback mechanism;
a second feedback mechanism;
a sensor configured to identify a position of a pupil of an eye; and
a controller communicatively coupled to the sensor, the plurality of visible indicia, and the second feedback mechanism, the controller configured to:
receive a first signal from the sensor, the first signal indicative of a gaze direction of the pupil,
identify, based on the first signal, a gazed upon visible indicia, the gazed upon visible indicia being one of the plurality of visible indicia that most closely corresponds to the gaze direction,
map the gazed upon visible indicia to a virtual reference frame comprising a plurality of members of a set,
identify a provisional input selection from the plurality of members of the set,
cause at least one of the first feedback mechanism and the second feedback mechanism to signal that the provisional input selection has been made,
update the virtual reference frame if the provisional input selection is not one of a group of predetermined final input selections, and
transmit a second signal to the second feedback mechanism or to another device to initiate an action based on the provisional input selection if the provisional input selection is one of the group of predetermined final input selections.

### Claim 2 (depends on 1)

The system of claim 1, wherein the signal that the provisional input selection has been made includes at least one of changing a brightness of at least one of the plurality of visible indicia and blinking at least one of the plurality of visible indicia.

### Claim 3 (depends on 2)

The system of claim 2, wherein changing the brightness of at least one of the plurality of visible indicia includes gradually increasing the brightness of the at least one of the plurality of visible indicia.

### Claim 4 (depends on 1)

The system of claim 1, wherein the support structure includes a frame which supports an infrared light source for illuminating the eye.

### Claim 5 (depends on 1)

The system of claim 1, wherein the second feedback mechanism includes at least one of a speaker, an earpiece, a mounted heads-up display, and a visual display.

### Claim 6 (depends on 1)

The system of claim 1, wherein the support structure includes glasses for being worn by the user.

### Claim 7 (depends on 6)

The system of claim 6, further comprising a single camera located external to the support structure.

### Claim 8 (depends on 7)

The system of claim 7, wherein the support structure includes an arm that extends in front of the glasses and supports the camera.

### Claim 9 (depends on 1)

The system of claim 1, wherein the second feedback mechanism includes a visual display, and wherein the visual display is configured to display a representation of the virtual reference frame.

### Claim 10 (depends on 1)

The system of claim 1, wherein the group of final input selections includes at least one of a letter, a number, a word, a phrase, and a sentence.

### Claim 11 (depends on 1)

The system of claim 1, wherein the controller is configured to transmit a control signal to another device if the provisional input selection is one of the group of final input selections, and wherein the another device is selected from the group consisting of a television, a medical support device, and a light.

### Claim 12 (depends on 1)

The system of claim 1, wherein the controller is configured to predict user selections using word and sentence completion techniques.

### Claim 13

A method, comprising:
receiving, by a processor, a first signal from a sensor, the first signal indicative of a gaze direction of a pupil of an eye of a user gazing at at least one of a plurality of visible indicia located at a periphery of the user's field of vision, the plurality of visible indicia establishing a physical reference frame having a plurality of colors and configured to provide a first feedback mechanism;
identify, based on the first signal, a gazed upon visible indicia, the gazed upon visible indicia being one of the plurality of visible indicia;
mapping, by the processor, the gazed upon visible indicia to a virtual reference frame comprising a plurality of members of a set;
identifying a provisional input selection from the plurality of members of the set;
causing at least one of the first feedback mechanism and a second feedback mechanism to signal that the provisional input selection has been made;
updating the virtual reference frame if the provisional input selection is not one of a group of predetermined final input selections; and
transmitting a second signal to the second feedback mechanism or to another device to initiate an action based on the provisional input selection if the provisional input selection is one of the group of predetermined final input selections.

### Claim 14 (depends on 13)

The method of claim 13, wherein the first feedback mechanism signals that the provisional input selection has been identified.

### Claim 15 (depends on 14)

The method of claim 14, further comprising adjusting a brightness of at least one of the plurality of visible indicia in response to identifying the provisional input selection.

### Claim 16 (depends on 15)

The method of claim 15, wherein adjusting the brightness of at least one of the plurality of visible indicia includes gradually increasing the brightness.

### Claim 17 (depends on 14)

The method of claim 14, further comprising causing at least one of the plurality of visible multi-colored indicia to blink in response to identifying the provisional input selection.

### Claim 18 (depends on 13)

The method of claim 13, wherein the second signal is transmitted to the second feedback mechanism, the second feedback mechanism includes a controllable device, and the method further comprises causing the controllable device to perform a function in response to receiving the second signal.

### Claim 19

A system, comprising:
a support structure for supporting a plurality of visible indicia such that the plurality of visible indicia is located at a periphery of a user's field of vision, the plurality of visible indicia collectively establishing an external reference frame;
at least one sensor configured to identify a position of a pupil of an eye; and
a controller communicatively coupled to the at least one sensor and the plurality of visible indicia, the controller configured to:
display a virtual reference frame on a visual display, the virtual reference frame comprising a matrix comprising a plurality of cells of alphabetic characters,
receive a first signal from the at least one sensor, the first signal indicative of a first gaze direction of the pupil gazing at at least one of the plurality of visible indicia,
map the first position of the pupil relative to the external reference frame to a row or column of the virtual reference frame,
receive a second signal from the at least one sensor, the second signal indicative of a second position of the pupil gazing at at least one of the plurality of visible indicia;
map the second position of the pupil relative to the external reference frame to the other of a row or column of the virtual reference frame; and
identify a first input selection based on the mapping of the first position and the second position to the virtual reference frame, the first input selection being, of the plurality of cells, the cell corresponding to the row and column selected by the user.