# US 10,481,699 B2 · Armband for tracking hand motion using electrical impedance measurement

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,481,699 B2
- Document ID: US10481699B2
- Status: Granted
- Kind: B2
- Application number: 15/661,317
- Filed: 2017-07-27
- Published: 2019-01-31
- Granted: 2019-11-19
- Earliest claimed: 2017-07-27
- Assignees: Facebook Technologies, LLC
- Applicants: Facebook Technologies, LLC
- Inventors: Beipeng Mu; Renzo De Nardi; Richard Andrew Newcombe; Raymond King; Evan Paul Gander; Robert Y. Wang
- Web page: https://patentgrep.com/patent/US10481699B2

## CPC

- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/015: {Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection}
- G06F 3/016: {Input arrangements with force or tactile feedback as computer generated output to the user}
- G06F 3/0308: {comprising a plurality of distinctive and separately oriented light emitters or reflectors associated to the pointing device, e.g. remote cursor controller with distinct and separately oriented LEDs at the tip whose radiations are captured by a photo-detector associated to the screen}
- G06F 3/0346: with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
- G06T 7/74: {involving reference images or patches}
- G06V 40/28: {Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113)}
- G06T 2207/20081: Training; Learning
- G06T 2207/30196: Human being; Person
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A system includes a wearable device including sensors arranged at different locations on the wearable device. Each sensor measures electrical signals transmitted from a wrist or arm of a user. A position computation circuit is coupled to the sensors. The position computation circuit computes, using information derived from the electrical signals with a machine learning model, an output that describes a hand position of a hand of the wrist or arm of the user.

## Claims

### Claim 1

A system, comprising:
a wearable device, comprising sensors arranged in a two-dimensional pattern on the wearable device, each sensor configured to measure electrical signals transmitted through a wrist or arm of a user, the electrical signals comprising information of values of an electrical impedance; and
a position computation circuit coupled to the sensors, the position computation circuit configured to use a machine learning model to receive the values of the electrical impedance derived from the electrical signals measured by the sensors arranged in the two-dimensional pattern and transmitted from the wrist or arm of the user to compute an output that describes a hand position of a hand of the wrist or arm of the user.

### Claim 2 (depends on 1)

The system of claim 1, wherein the position computation circuit is located on or within the wearable device.

### Claim 3 (depends on 1)

The system of claim 1, wherein the position computation circuit is located on or within a host system external to the wearable device.

### Claim 4 (depends on 1)

The system of claim 1, wherein each sensor comprises an electrode and a conductive agent located between the electrode and the wrist or arm of the user, each sensor further configured to transmit an alternating current (AC) signal, a direct current (DC) signal, or a wide-bandwidth AC signal including a plurality of frequencies into the wrist or arm of the user.

### Claim 5 (depends on 1)

The system of claim 1, wherein each electrical signal comprises one of:
a voltage and a phase of the voltage;
an electric current and a phase of the electric current; or
a magnitude of a DC signal.

### Claim 6 (depends on 1)

The system of claim 1, wherein the two-dimensional pattern is a grid array or a checkerboard pattern.

### Claim 7 (depends on 1)

The system of claim 1, wherein the machine learning model is configured to use information derived from the electrical signals to compute the output, the information comprises one or more of:
aggregate values based on the electrical impedance, the electrical impedance measured between each pair of the sensors, the electrical impedance determined based on probe signals transmitted into the wrist or arm of the user by the pair of sensors and the electrical signals;
a shape of a wave of the electrical signals;
a frequency-domain representation of the electrical signals; and
a time-domain sample of the electrical signals.

### Claim 8 (depends on 1)

The system of claim 1, wherein the position computation circuit is further configured to extract features from information derived from the electrical signals, the features comprising one or more of:
angles between joints defining the hand position of a hand of the wrist or arm of the user; and
a reduced representation of a change in the hand position of a hand of the wrist or arm of the user, the reduced representation defining a difference between a present hand position of the hand of the wrist or arm of the user and a previous hand position of the hand of the wrist or arm of the user.

### Claim 9 (depends on 1)

The system of claim 1, wherein each sensor is further configured to transmit a probe signal into the wrist or arm of the user by varying one or more of:
a time period of the probe signal;
an amplitude of the probe signal; and
a phase of the probe signal.

### Claim 10 (depends on 1)

The system of claim 1, wherein a first sensor is configured to transmit a probe signal into the wrist or arm of the user by staggering transmission of the probe signal with respect to transmission of other probe signals by other sensors.

### Claim 11 (depends on 1)

The system of claim 1, further comprising:
an inertial measurement unit configured to generate inertial signals corresponding to movement of the wearable device and the user's arm, wherein the position computation circuit is further configured to compute, using information derived from the inertial signals with the machine learning model, the output that describes the hand position of the hand of the wrist or arm of the user.

### Claim 12 (depends on 11)

The system of claim 11, wherein the inertial measurement unit comprises one or more of:
a gyroscope;
an accelerometer; and
a magnetometer.

### Claim 13 (depends on 1)

The system of claim 1, further comprising:
one or more cameras configured to generate image signals of the hand of the wrist or arm of the user from one or more angles, wherein the position computation circuit is further configured to compute, using information derived from the image signals with the machine learning model, the output that describes the hand position of the hand of the wrist or arm of the user.

### Claim 14 (depends on 13)

The system of claim 13, wherein one of the cameras is a depth camera, a Red-Green-Blue (RGB) camera, an Infrared camera, or a camera mounted on a head-mounted display (HMD).

### Claim 15 (depends on 1)

The system of claim 1, further comprising:
an inertial measurement unit configured to generate inertial signals corresponding to movement of the wearable device, wherein the position computation circuit is further configured to train the machine learning model to generate, using information derived from the inertial signals, the output that describes the hand position of the hand of the wrist or arm of the user.

### Claim 16 (depends on 1)

The system of claim 1, further comprising:
one or more cameras configured to generate image signals of the hand of the wrist or arm of the user, wherein the position computation circuit is further configured to train the machine learning model to generate, using information derived from the image signals with the machine learning model, the output that describes the hand position of the hand of the wrist or arm of the user.

### Claim 17 (depends on 1)

The system of claim 1, wherein the computed output further describes forces exerted by the hand of the wrist or arm of the user on objects touching the hand of the wrist or arm of the user.

### Claim 18 (depends on 1)

The system of claim 1, wherein the position computation circuit is further configured to:
receive image signals from a camera mounted on an HMD;
determine comparison signals by comparing the image signals with the computed output; and
transmit the comparison signals to a host system.

### Claim 19 (depends on 1)

The system of claim 1, wherein the computed output comprises parameters of a hand shape model, the parameters corresponding to:
a plurality of joints of the hand of the wrist or arm of the user;
a plurality of edges between pairs of the joints;
a plurality of angles between pairs of the edges;
a mesh comprising a plurality of vertices and for each vertex, a distance between the vertex and one or more joints.

### Claim 20

An HMD, comprising:
a position computation circuit coupled to sensors arranged in a two-dimensional pattern on a wearable device, each sensor configured to measure electrical signals transmitted from a wrist or arm of a user, the electrical signals comprising information of values of an electrical impedance, the position computation circuit configured to use a machine learning model to receive the values of the electrical impedance derived from the electrical signals measured by the sensors arranged in the two-dimensional pattern and transmitted from the wrist or arm of the user to compute an output that describes a hand position of a hand of the wrist or arm of the user; and
a display panel configured to receive the computed output from the position computation circuit.

### Claim 21 (depends on 20)

The HMD of claim 20, further comprising:
a camera configured to generate image signals of the hand of the wrist or arm of the user, wherein the position computation circuit is further configured to compute, using information derived from the image signals with the machine learning model, the output that describes the hand position of the hand of the wrist or arm of the user.

### Claim 22 (depends on 21)

The system of claim 21, wherein the camera is a depth camera, a Red-Green-Blue (RGB) camera, an Infrared camera, or a camera mounted on the HMD.

### Claim 23 (depends on 20)

The HMD of claim 20, further comprising:
a camera configured to generate image signals of the hand of the wrist or arm of the user, wherein the position computation circuit is further configured to train the machine learning model to generate, using information derived from the image signals with the machine learning model, the output that describes the hand position of the hand of the wrist or arm of the user.

### Claim 24 (depends on 23)

The HMD of claim 23, wherein the position computation circuit is further configured to:
determine comparison signals by comparing the image signals with the computed output; and
transmit the comparison signals to a host system.

### Claim 25

A method, comprising:
receiving training electrical signals from sensors arranged in a two-dimensional pattern on a wearable device, the training electrical signals transmitted from a wrist or arm of a user;
extracting training features from the training electrical signals;
training a machine learning model using the extracted training features to determine a hand position of a hand of the wrist or arm of the user;
receiving electrical signals from the sensors, the electrical signals transmitted from the wrist or arm of a user, the electrical signals comprising information of values of an electrical impedance;
extracting features from the electrical signals, the features comprising the values of the electrical impedance;
transmitting the features, which are extracted from the electrical signals measured by the sensors arranged in the two-dimensional pattern, to the machine learning model to compute an output that describes a hand position of the hand of the wrist or arm of the user; and
transmitting the output to a host system.

### Claim 26 (depends on 25)

The method of claim 25, further comprising:
transmitting an AC signal, a DC signal, or a wide-bandwidth AC signal comprising a plurality of frequencies into the wrist or arm of the user.

### Claim 27 (depends on 25)

The method of claim 25, wherein the features extracted from the electrical signals comprise one or more of
aggregate values based on the electrical impedance, the electrical impedance measured between each pair of the sensors, the electrical impedance determined based on probe signals transmitted into the wrist or arm of the user by the pair of sensors and the electrical signals;
a shape of a wave of the electrical signals;
a frequency-domain representation of the electrical signals; and
a time-domain sample of the electrical signals.

### Claim 28 (depends on 27)

The method of claim 27, wherein the features extracted from the electrical signals further comprises one or more of:
angles between joints defining the hand position of the hand of the wrist or arm of the user; and
a reduced representation of a change in the hand position of the hand of the wrist or arm of the user, the reduced representation defining a difference between a present hand position of the hand of the wrist or arm of the user from a previous hand position of the hand of the wrist or arm of the user.

### Claim 29 (depends on 25)

The method of claim 25, further comprising transmitting a probe signal into the wrist or arm of the user by varying one or more of:
a time period of the probe signal;
an amplitude of the probe signal; and
a phase of the probe signal.

### Claim 30 (depends on 25)

The method of claim 25, further comprising:
transmitting, by a first sensor, a probe signal into the wrist or arm of the user by staggering transmission of the probe signal with respect to transmission of other probe signals transmitted by other sensors.

### Claim 31 (depends on 25)

The method of claim 25, further comprising:
generating, by an inertial measurement unit, inertial signals corresponding to movement of the wearable device; and
computing, using information derived from the inertial signals with the machine learning model, the output that describes the hand position of the hand of the wrist or arm of the user.

### Claim 32 (depends on 25)

The method of claim 25, further comprising:
generating, by one or more cameras, image signals of the hand of the wrist or arm of the user; and
computing, using information derived from the image signals with the machine learning model, the output that describes the hand position of the hand of the wrist or arm of the user.

### Claim 33 (depends on 25)

The method of claim 25, further comprising:
generating, by an inertial measurement unit, inertial signals corresponding to movement of the wearable device; and
training the machine learning model to generate, using information derived from the inertial signals, the output that describes the hand position of the hand of the wrist or arm of the user.

### Claim 34 (depends on 25)

The method of claim 25, further comprising:
generating, by one or more cameras, image signals of the hand of the wrist or arm of the user; and
training the machine learning model to generate, using information derived from the image signals, the output that describes the hand position of the hand of the wrist or arm of the user.

### Claim 35 (depends on 25)

The method of claim 25, further comprising:
receiving, from one or more cameras, image signals of the hand of the wrist or arm of the user;
determining comparison signals by comparing the image signals with the computed output; and
transmitting the comparison signals to a host system.