# US 2020/0401215 A1 · Control of a computer via distortions of facial geometry

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0401215 A1
- Document ID: US20200401215A1
- Status: Published application
- Kind: A1
- Application number: 16/990,917
- Filed: 2020-08-11
- Published: 2020-12-24
- Earliest claimed: 2015-01-31
- Inventors: Brian Lee Moffat; Rin In Chen
- Web page: https://patentgrep.com/patent/US20200401215A1

## CPC

- G06F 3/012: {Head tracking input arrangements}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/0233: {Character input methods}
- G06F 3/0304: {Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042)}
- G06V 40/11: {Hand-related biometrics; Hand pose recognition}
- G06V 40/113: {Recognition of static hand signs}
- G06V 40/175: {Static expression}
- G06V 40/176: {Dynamic expression}
- G06V 40/193: {Preprocessing; Feature extraction}
- G06V 40/28: {Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113)}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A system which, with data provided by one or more sensors, detects a user's alteration of the geometries of parts of his face, head, neck, and/or shoulders. It determines the extent of each alteration and normalizes it with respect to the maximum possible range of each alteration so as to assign to each part-specific alteration a numeric score indicative of its extent. The normalized part-specific scores are combined so as to produce a composite numeric code representative of the complete set of simultaneously-executed geometric alterations. Each composite code is translated, or interpreted, relative to an appropriate context defined by an embodiment, an application executing on an embodiment, or by the user. For example, each composite code might be interpreted as, or assigned to, a specific alphanumeric letter, a color, a musical note, etc. Through the use of this system, a user may communicate data and/or commands to a computerized device, while retaining full use of his hands and his voice for other tasks, and while being free to focus his visual attention on something other than the system.

## Claims

### Claim at position 1

1-222. (canceled)

### Claim 223

A computer symbol entry system comprising:
a video camera adapted to generate a video data stream;
a digitally encoded mapping of a plurality of facial and upper torso distortions to alphanumeric characters; and
a software program configured to encode facial and upper torso distortions from the video data stream, and correlate alphanumeric characters from the facial and upper torso distortions using the digitally encoded mapping.

### Claim 224 (depends on 223)

The computer symbol entry system of claim 223, wherein the software program encodes facial and upper torso distortions by encoding head tilt distortions.

### Claim 225 (depends on 203)

The computer symbol entry system of claim 203, wherein the software program encodes facial and upper torso distortions by encoding shoulder position distortions.

### Claim 226 (depends on 203)

The computer symbol entry system of claim 203, wherein the software program encodes facial and upper torso distortions by encoding eyebrow distortions.

### Claim 227 (depends on 203)

The computer symbol entry system of claim 203, wherein the software program encodes facial and upper torso distortions by encoding cheek distortions.

### Claim 228

A computer controller comprising:
a video camera;
a software program configured to read a digital output of the video camera and interpret a sequence of images within the digital output to identify distortions of facial features in the sequence of images by computing numerical representations of facial features to produce a representation set and computing a function of the representation set to produce a distortion representation comprising a symbol;
wherein the software program executes a computer command when the software program identifies a distortion of facial features in the sequence of images.

### Claim 229

A computer controller comprising:
a video camera; and
a hardware circuit configured to interpret a sequence of images within the digital output of the video camera to identify distortions of facial features in the sequence of images by computing numerical representation of facial features to produce a representation set and computing a function of the representation set to produce a distortion representation that is a symbol;
wherein the hardware circuit sends an electronic control signal to a computer when the hardware circuit identifies a distortion of facial features in the sequence of images.

### Claim 230

A system comprising:
at least one sensor configured to assess a status of a user's face, head, neck, and shoulders, and generating a signal indicative of said status;
a conversion circuit receiving the signal and generating in response thereto a user-selected symbolic value; and
a processor receiving the user-selected symbolic value,
wherein, based on the user-selected symbolic value, the system alters at least one component of at least one computing device.

### Claim 231

A method for controlling a computing device using facial distortions, comprising:
assessing, using at least one sensor, an extent to which specific muscles, located, at least partially, within a face, head, neck, and shoulders, of a user are contracted and thereby derive a plurality of muscle-specific statuses;
quantizing each of the muscle-specific statuses;
combining the quantized muscle-specific statuses into a composite number unique to each unique combination of quantized muscle statuses;
converting the composite number into a context-specific user-selected symbolic value; and
using the context-specific user-selected value to modify at least one bit of digital information.
