# US 2018/0024634 A1 · Methods and apparatus for inferring user intent based on neuromuscular signals

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2018/0024634 A1
- Document ID: US20180024634A1
- Status: Published application
- Kind: A1
- Application number: 15/659,018
- Filed: 2017-07-25
- Published: 2018-01-25
- Earliest claimed: 2016-07-25
- Assignees: CTRL-labs Corporation
- Inventors: Patrick Kaifosh; Timothy Machado; Thomas Reardon; Erik Schomburg
- Web page: https://patentgrep.com/patent/US20180024634A1

## CPC

- G06F 3/015: {Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection}
- A61B 5/395: Details of stimulation, e.g. nerve stimulation to elicit EMG response
- A61B 5/7267: {involving training the classification device}
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/0499: Feedforward networks
- G06N 3/09: Supervised learning
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G06N 20/00: Machine learning
- A61B 5/7282: {Event detection, e.g. detecting unique waveforms indicative of a medical condition (cough events A61B5/0823; seizures A61B5/4094; sleep apnoea A61B5/4818)}
- G06N 3/045: Combinations of networks
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 20/10: using kernel methods, e.g. support vector machines [SVM]
- G16H 50/70: for mining of medical data, e.g. analysing previous cases of other patients
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA

## Abstract

Methods and system for predicting the onset of a motor action using neuromuscular signals. The system comprises a plurality of sensors configured to continuously record a plurality of neuromuscular signals from a user and at least one computer processor programmed to provide as input to a trained statistical model, the plurality of neuromuscular signals or information based on the plurality of neuromuscular signals, predict, based on an output of the trained statistical model, whether an onset of a motor action will occur within a threshold amount of time; and send a control signal to at least one device based, at least in part, on the output probability, wherein the control signal is sent to the at least one device prior to completion of the motor action by the user.

## Claims

### Claim 1

A control system, comprising:
a plurality of sensors configured to continuously record a plurality of neuromuscular signals from a user; and
at least one computer processor programmed to:
provide as input to a trained statistical model, the plurality of neuromuscular signals and/or information based on the plurality of neuromuscular signals;
predict, based on an output of the trained statistical model, whether an onset of a motor action will occur within a threshold amount of time; and
send a control signal to at least one device based, at least in part, on the output probability, wherein the control signal is sent to the at least one device prior to completion of the motor action by the user.

### Claim 2 (depends on 1)

The control system of claim 1, wherein the output of the trained statistical model comprises a probability that the onset of the motor action will occur within the threshold amount of time, and
wherein predicting that an onset of a motor action will occur within a threshold amount of time comprises determining whether the probability is greater than an operating threshold.

### Claim 3 (depends on 2)

The control system of claim 2, wherein the at least one computer processor is further programmed to determine the operating threshold based, at least in part, on an analysis of data obtained from neuromuscular signals recorded by at least some of the plurality of sensors.

### Claim 4 (depends on 3)

The control system of claim 3, wherein determining the operating threshold comprises:
determining a mean anticipation time based on the data; and
determining the operating threshold based, at least in part, on the mean anticipation time.

### Claim 5 (depends on 2)

The control system of claim 2, wherein the at least one computer processor is further programmed to receive at least one performance characteristic for a task to be performed, and wherein the operating threshold is determined based, at least in part, on the received at least one performance characteristic.

### Claim 6 (depends on 5)

The control system of claim 5, wherein the at least one performance characteristic includes a performance characteristic selected from the group consisting of a false positive rate, a false negative rate, a true positive rate, a true negative rate, a positive predictive value, a negative predictive value, and a mean anticipation time.

### Claim 7 (depends on 1)

The control system of claim 1, wherein the output of the trained statistical model comprises a set of probabilities, each probability in the set indicating a probability that the onset of a respective motor action of a plurality of motor actions will occur within the threshold amount of time, and
wherein predicting that an onset of a motor action will occur within a threshold amount of time comprises determining whether any of the probabilities in the set of probabilities is greater than an operating threshold stored by the system.

### Claim 8 (depends on 1)

The control system of claim 1, wherein the output of the trained statistical model comprises a probability that the onset of a sequence of motor actions will occur within the threshold amount of time, and
wherein predicting that an onset of a motor action will occur within a threshold amount of time comprises determining whether the probability is greater than an operating threshold.

### Claim 9 (depends on 1)

The control system of claim 1, wherein the at least one computer processor is further programmed to retrain the trained statistical model based, at least in part, on at least some of the plurality of neuromuscular signals.

### Claim 10 (depends on 1)

The control system of claim 1, wherein the trained statistical model comprises a recurrent neural network.

### Claim 11 (depends on 1)

The control system of claim 1, wherein the at least one computer processor is further programmed to:
determine a set of features based on the plurality of neuromuscular signals; and
provide as input to a trained statistical model, the set of features as information based on the plurality of neuromuscular signals,
wherein predicting, based on an output of the trained statistical model, that an onset of a motor action will occur within a threshold amount of time comprises classifying the set of features using the trained statistical model.

### Claim 12 (depends on 1)

The control system of claim 1, wherein the plurality of sensors are arranged on a wearable device configured to be worn on or around a body part of the user.

### Claim 13 (depends on 12)

The control system of claim 12, wherein the wearable device comprises a flexible or elastic band configured to be worn around the body part of the user.

### Claim 14 (depends on 13)

The control system of claim 13, wherein the wearable device comprises an armband configured to be worn around an arm of the user.

### Claim 15 (depends on 1)

The control system of claim 1, further comprising hardware circuitry and/or software configured to process the neuromuscular signals prior to being provided as input to the trained statistical model, wherein processing the neuromuscular signals comprises processing selected from the group consisting of amplification, filtering, and rectification.

### Claim 16 (depends on 1)

The control system of claim 1, wherein sending the control signal to the at least one device prior to completion of the motor action by the user comprises sending the control signal at least 25 ms prior to the completion of the motor action by the user.

### Claim 17 (depends on 1)

The control system of claim 1, wherein the at least one device comprises at least one device selected from the group consisting of a video game system, a consumer electronics device, a manned vehicle, an unmanned vehicle, a robot, and a weapon.

### Claim 18 (depends on 1)

The control system of claim 1, wherein the plurality of sensors comprises sensors selected from the group consisting of electromyography (EMG) sensors, mechanomyography (MMG) sensors, and sonomyography (SMG) sensors.

### Claim 19

A method of predicting an onset of a motor action, the method comprising
providing, as input to a trained statistical model, a plurality of neuromuscular signals recorded from a plurality of sensors arranged on or around a part of a user's body;
predicting, using at least one computer processor, whether an onset of a motor action will occur within a threshold amount of time; and
sending a control signal to at least one device based, at least in part, on the output probability, wherein the control signal is sent to the at least one device prior to completion of the motor action by the user.

### Claim 20

A computer system for training a statistical model to predict an onset of a motor task based, at least in part, on neuromuscular signal data, the computer system comprising:
an input interface configured to:
receive the neuromuscular signal data recorded during performance of a motor action performed by one or more users;
receive result data indicating an outcome of the motor action performed by the one or more users; and
at least one storage device configured to store a plurality of instructions that, when executed by at least one computer processor perform a method of:
generating training data based, at least on part, on the received neuromuscular signal data and the received result data;
training the statistical model using at least some of the generated training data to output a trained statistical model;
determining an operating threshold used to interpret output of the trained statistical model; and
storing, by the at least one storage device, the trained statistical model and the operating threshold, wherein the trained statistical model is configured to predict the onset of a motor task prior to completion of the motor task by a user.