# US 10,503,254 B2 · Motion and gesture input from a wearable device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,503,254 B2
- Document ID: US10503254B2
- Status: Granted
- Kind: B2
- Application number: 15/914,838
- Filed: 2018-03-07
- Published: 2018-07-12
- Granted: 2019-12-10
- Earliest claimed: 2015-09-25
- Assignees: Apple Inc.
- Applicants: Apple Inc.
- Inventors: Nicholas Paul Joseph Allec; Xiaoyi Mu
- Web page: https://patentgrep.com/patent/US10503254B2

## CPC

- G06F 3/014: {Hand-worn input/output arrangements, e.g. data gloves}
- G06F 3/015: {Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/0304: {Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042)}
- G06F 3/0308: {comprising a plurality of distinctive and separately oriented light emitters or reflectors associated to the pointing device, e.g. remote cursor controller with distinct and separately oriented LEDs at the tip whose radiations are captured by a photo-detector associated to the screen}
- G06F 18/23: Clustering techniques
- G06V 40/28: {Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113)}
- G06F 2218/12: Classification; Matching
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

This disclosure relates to detecting hand gesture input using an electronic device, such as a wearable device strapped to a wrist. The device can have multiple photodiodes, each sensing light at a different position on a surface of the device that faces skin of a user. Examples of the disclosure detect hand gestures by recognizing patterns in sensor data that are characteristic of each hand gesture, as the tissue expands and contracts and anatomical features in the tissue move during the gesture.

## Claims

### Claim 1

A device comprising:
a housing, configured to be strapped to a wrist of a user with a surface of the housing proximate to one side of the wrist;
a plurality of light emitters disposed within the housing and configured to emit light through the surface toward a tissue of the user;
a plurality of photodiodes disposed within the housing and configured to:
receive at least a portion of a reflection of the emitted light through the surface; and
generate sensor data indicative of movement of anatomic features in the tissue of the user;
a plurality of channels, each channel associated with one of the plurality of light emitters and one of the plurality of photodiodes and further associated with a signal pattern including the sensor data from the respective photodiode; and
logic disposed within the housing and configured to:
receive the signal patterns of the plurality of channels,
determine signal characteristics from the signal patterns,
determine a difference in the signal characteristics of at least two of the plurality of channels, and
determine a hand gesture from the determined difference in the signal characteristics.

### Claim 2 (depends on 1)

The device of claim 1, wherein the signal patterns from at least some of the plurality of channels are associated with a hand clench of the user, and the signal patterns from others of the plurality of channels are associated with a hand flap of the user.

### Claim 3 (depends on 1)

The device of claim 1, wherein the plurality of light emitters includes two light emitters, the plurality of photodiodes includes two photodiodes, and the plurality of channels includes eight channels.

### Claim 4 (depends on 1)

The device of claim 1, wherein the determination of the signal characteristics of the signal patterns includes determining a relationship between a peak and a trough of at least one of the signal patterns.

### Claim 5 (depends on 4)

The device of claim 4, wherein at least some of the signal characteristics include the trough occurring before the peak in the respective signal pattern, and the relationship is identified by a sign indicative of the trough occurring before the peak.

### Claim 6 (depends on 4)

The device of claim 4, wherein at least some of the signal characteristics include the peak occurring before the trough in the respective signal pattern, and the relationship is identified by a sign indicative of the peak occurring before the trough.

### Claim 7 (depends on 1)

The device of claim 1, wherein the determination of the signal characteristics of the signal patterns includes determining a time difference between a peak and a trough in a signal pattern.

### Claim 8 (depends on 1)

The device of claim 1, wherein the determination of the signal characteristics of the signal patterns includes determining a period between peaks in a signal pattern.

### Claim 9 (depends on 1)

The device of claim 1, wherein the determination of the signal characteristics of the signal patterns includes determining a phase between peaks in a signal pattern.

### Claim 10 (depends on 1)

The device of claim 1, wherein the sensor data is divided into a plurality of frames and the logic is further configured to:
analyze one or more of the plurality of frames and extract frequency information as additional signal characteristics used in the determination of the hand gesture.

### Claim 11 (depends on 1)

The device of claim 1, further comprising:
one or more of a force sensor and an accelerometer, wherein the force sensor is configured to measure a force and generate force sensor data, and the accelerometer is configured to measure an acceleration and generate accelerometer data,
wherein the logic is further configured to:
receive the force sensor data, the accelerometer data, or both, and
further determine the signal characteristics from the received force sensor data, the received accelerometer data, or both.

### Claim 12

A method of determining a hand gesture, the method comprising:
emitting light from a plurality of light emitters disposed in a housing strapped to a wrist of a user, the housing having a surface proximate to one side of the wrist, and the light emitted through the surface toward a tissue of the user;
receiving at least a portion of a reflection of the emitted light through the surface, the portion of the reflection being received using a plurality of photodiodes disposed within the housing;
generating sensor data from the received light, the sensor data indicative of movement of anatomic features in the tissue of the user;
associating the plurality of light emitters and plurality of photodiodes with a plurality of channels;
associating signal patterns included in the sensor data to the plurality of channels;
determining signal characteristics from the signal patterns;
determining a difference in the signal characteristics of at least two of the plurality of channels; and
determining the hand gesture from the determined difference in the signal characteristics.

### Claim 13 (depends on 12)

The method of claim 12, wherein the association of the signal patterns to the plurality of channels includes:
associating the signal patterns from at least some of the plurality of channels with a hand clench of the user; and
associating the signal patterns from others of the plurality of channels with a hand flap of the user.

### Claim 14 (depends on 12)

The method of claim 12, wherein the determination of the signal characteristics of the signal patterns includes determining a relationship between a peak and a trough of a signal pattern.

### Claim 15 (depends on 14)

The method of claim 14, wherein the relationship includes a sign indicative of a relative order of the peak and the trough in the signal pattern or a time difference between the peak and the trough in the signal pattern.

### Claim 16 (depends on 12)

The method of claim 12, further comprising:
performing one or more of:
measuring a force using a force sensor and generating force sensor data indicative of the measured force, and
measuring an acceleration using an accelerometer and generating accelerometer data indicative of the measured acceleration; and
further determining the signal characteristics from the received force sensor data, the received accelerometer data, or both.

### Claim 17 (depends on 12)

The method of claim 12, further comprising: filtering out information from the sensor data before the determination of the signal characteristics.

### Claim 18 (depends on 17)

The method of claim 17, wherein the information includes heart rate frequencies.