# US 11,906,540 B1 · Automatic detection of falls using hybrid data processing approaches

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,906,540 B1
- Document ID: US11906540B1
- Status: Granted
- Kind: B1
- Application number: 17/515,086
- Filed: 2021-10-29
- Granted: 2024-02-20
- Earliest claimed: 2020-10-30
- Assignees: Bby Solutions, Inc.
- Applicants: Bby Solutions, Inc.
- Inventors: Joseph Gwin; Hyoki Lee; Ramin Oftadeh
- Web page: https://patentgrep.com/patent/US11906540B1

## CPC

- G01P 15/18: in two or more dimensions
- G08B 21/0446: {worn on the body to detect changes of posture, e.g. a fall, inclination, acceleration, gait}
- A61B 5/1117: {Fall detection}
- A61B 5/6802: {Sensor mounted on worn items}
- A61B 5/681: {Wristwatch-type devices}
- A61B 5/6898: {Portable consumer electronic devices, e.g. music players, telephones, tablet computers}
- G01P 13/00: Indicating or recording presence, absence, or direction, of movement (electric switches H01H; counting moving objects G06M7/00)
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/08: Learning methods
- G06N 5/01: Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G06N 20/20: Ensemble learning
- G08B 21/043: {detecting an emergency event, e.g. a fall}
- G08B 29/186: {Fuzzy logic; neural networks}
- A61B 2562/0219: Inertial sensors, e.g. accelerometers, gyroscopes, tilt switches
- G01P: MEASURING LINEAR OR ANGULAR SPEED, ACCELERATION, DECELERATION, OR SHOCK; INDICATING PRESENCE, ABSENCE, OR DIRECTION, OF MOVEMENT (measuring or recording blood flow A61B5/02, A61B8/06; monitoring speed or deceleration of electrically-propelled vehicles B60L3/00; vehicle lighting systems adapted to indicate speed B60Q1/54; determining position or course in navigation, measuring ground distance in geodesy or surveying G01C; combined measuring devices for measuring two or more variables of movement G01C23/00; measuring velocity of sound G01H; measuring velocity of light G01J7/00; measuring direction or velocity of solid objects by reception or emission of radiowaves or other waves and based on propagation effects, e.g. Doppler effect, propagation time, direction of propagation, G01S; measuring speed of nuclear radiation G01T; measuring acceleration of gravity G01V; {measuring or recording the speed of trains B61L23/00; speed indicators incorporated in motor vehicles B60K35/00; measuring frequency or phase G01R; traffic control G08G})
- G08B: SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Various approaches for automated fall detection, implemented in wearables and other device form factors of a personal emergency response system (PERS) are disclosed. In an example, a hybrid approach for fall detection includes: identifying a potential fall event from three-dimensional motion data of a human subject, using filtering rules; evaluating the motion data with a machine learning model (e.g., a decision tree ensemble (DTE) model), to produce a first determination that a fall has occurred; evaluating the motion data with a deep learning neural network (e.g., recurrent neural network such as a gated recurrent unit (GRU)), to produce a second determination that a fall has occurred; classifying the potential fall event as a fall condition for the human subject, based on the first determination and the second determination; and outputting data to indicate the fall condition for the human subject.

## Claims

### Claim 1

A device, comprising:
a sensor configured to obtain three-dimensional motion data of a human subject, wherein the motion data includes sensor measurements collected in a subject time period and in an earlier time period before the subject time period; and
processing circuitry configured to:
identify a potential fall event from the motion data, based on one or more rules;
evaluate first samples of the motion data with a machine learning model, to produce a first determination that a fall has occurred during the potential fall event, wherein the first samples correspond to the sensor measurements collected in the subject time period, and wherein use of the machine learning model is invoked in response to the potential fall event identified using the one or more rules;
evaluate second samples of the motion data with a deep learning neural network, to produce a second determination that a fall has occurred during the potential fall event, wherein use of the deep learning neural network is invoked after the use of the machine learning model, in response to the first determination produced from the machine learning model indicating that the fall has occurred, wherein the second samples correspond to the sensor measurements collected in the subject time period and in the earlier time period;
classify the potential fall event as a fall condition for the human subject, based on the first determination produced from the machine learning model and the second determination produced from the deep learning neural network both indicating that the fall has occurred; and
output data to indicate the fall condition for the human subject.

### Claim 2 (depends on 1)

The device of claim 1, wherein the sensor is a tri-axial accelerometer, wherein the motion data is acceleration data which is downsampled, and wherein the motion data is collected for a predefined amount of time.

### Claim 3 (depends on 1)

The device of claim 1, wherein the one or more rules relate to acceleration, velocity, or variability of motion, wherein the one or more rules are applied to the motion data to identify the potential fall event.

### Claim 4 (depends on 1)

The device of claim 1, wherein the machine learning model is a decision tree ensemble model provided from a plurality of decision trees, and wherein the decision tree ensemble model is optimized during training with Bayesian optimization of parameter tuning and feature selection for the plurality of decision trees.

### Claim 5 (depends on 1)

The device of claim 1, wherein the deep learning neural network is a gated recurrent unit (GRU) model, wherein the GRU model is optimized during training using one or more of: data augmentation, down-sampling of data, variation of a number of layers of the GRU model, or adjusting weights used in training of the GRU model.

### Claim 6 (depends on 1)

The device of claim 1, the processing circuitry further configured to:
evaluate one or more samples of additional motion data with the deep learning neural network, after evaluation of the motion data with the deep learning neural network, wherein the additional motion data includes sensor measurements collected in a later time period after the subject time period;
wherein a counter is maintained to track consecutive fall determinations produced from the deep learning neural network, and wherein classification of the potential fall event as the fall condition for the human subject is further based on the counter exceeding a threshold.

### Claim 7 (depends on 1)

The device of claim 1, the processing circuitry further configured to:
perform fall verification of the potential fall event, before indication of the fall condition for the human subject, wherein the fall verification occurs based on a lack of activity from the human subject within a period of time after the potential fall event.

### Claim 8 (depends on 1)

The device of claim 1, the processing circuitry further configured to:
transmit a communication to a device external to the human subject, the communication including the data to indicate the fall condition for the human subject.

### Claim 9 (depends on 1)

The device of claim 1, wherein the device is a wearable adapted to be worn on a body of the human subject, and wherein the wearable is provided in a form factor provided from among a: watch, bracelet, or pendant.

### Claim 10 (depends on 1)

The device of claim 1, wherein the first determination includes a first probability that a fall has occurred, wherein the second determination includes a second probability that a fall has occurred, wherein use of the deep learning neural network is invoked in response to the first probability exceeding a first probability threshold that a fall has occurred, and wherein the potential fall event is classified as a fall condition for the human subject in response to the second probability exceeding a second probability threshold that a fall has occurred.

### Claim 11

A method for fall detection, performed by processing circuitry of a wearable device, comprising:
receiving three-dimensional motion data of a human subject, wherein the motion data includes sensor measurements collected in a subject time period and in an earlier time period before the subject time period;
identifying a potential fall event from the three-dimensional motion data of the human subject, the potential fall event identified based on one or more rules;
evaluating first samples of the motion data with a machine learning model, to produce a first determination that a fall has occurred during the potential fall event, wherein the first samples correspond to the sensor measurements collected in the subject time period, and wherein use of the machine learning model is invoked in response to the potential fall event identified using the one or more rules;
evaluating second samples of the motion data with a deep learning neural network, to produce a second determination that a fall has occurred during the potential fall event, wherein use of the deep learning neural network is invoked after the use of the machine learning model, in response to the first determination produced from the machine learning model indicating that the fall has occurred, wherein the second samples correspond to the sensor measurements collected in the subject time period and in the earlier time period;
classifying the potential fall event as a fall condition for the human subject, based on the first determination produced from the machine learning model and the second determination produced from the deep learning neural network both indicating that the fall has occurred; and
outputting data to indicate the fall condition for the human subject.

### Claim 12 (depends on 11)

The method of claim 11, wherein the motion data is provided from a tri-axial accelerometer, wherein the motion data is acceleration data which is downsampled, and wherein the motion data is collected for a predefined amount of time.

### Claim 13 (depends on 11)

The method of claim 11, wherein the one or more rules relate to acceleration, velocity, or variability of motion, wherein the one or more rules are applied to the motion data to identify the potential fall event.

### Claim 14 (depends on 11)

The method of claim 11, wherein the machine learning model is a decision tree ensemble model provided from a plurality of decision trees, and wherein the decision tree ensemble model is optimized during training with Bayesian optimization of parameter tuning and feature selection for the plurality of decision trees.

### Claim 15 (depends on 11)

The method of claim 11, wherein the deep learning neural network is a gated recurrent unit (GRU) model, wherein the GRU model is optimized during training using one or more of: data augmentation, down-sampling of data, variation of a number of layers of the GRU model, or adjusting weights used in training of the GRU model.

### Claim 16 (depends on 11)

The method of claim 11, further comprising:
evaluating one or more samples of additional motion data with the deep learning neural network, after evaluation of the motion data with the deep learning neural network, wherein the additional motion data includes sensor measurements collected in a later time period after the subject time period;
wherein a counter is maintained to track consecutive fall determinations produced from the deep learning neural network, and wherein classification of the potential fall event as the fall condition for the human subject is further based on the counter exceeding a threshold.

### Claim 17 (depends on 11)

The method of claim 11, further comprising:
performing fall verification of the potential fall event, before indication of the fall condition for the human subject, wherein the fall verification occurs based on a lack of activity from the human subject within a period of time after the potential fall event.

### Claim 18 (depends on 11)

The method of claim 11, further comprising:
transmitting a communication to a device external to the human subject, the communication including the data to indicate the fall condition for the human subject.

### Claim 19 (depends on 11)

The method of claim 11, wherein the first determination includes a first probability that a fall has occurred, wherein the second determination includes a second probability that a fall has occurred, wherein use of the deep learning neural network is invoked in response to the first probability exceeding a first probability threshold that a fall has occurred, and wherein the potential fall event is classified as a fall condition for the human subject in response to the second probability exceeding a second probability threshold that a fall has occurred.

### Claim 20

A non-transitory device-readable storage medium, the device-readable storage medium comprising instructions that, when executed by processor circuitry of an electronic device, causes the electronic device to perform operations that:
receive three-dimensional motion data of a human subject, wherein the motion data includes sensor measurements collected in a subject time period and in an earlier time period before the subject time period;
identify a potential fall event from the three-dimensional motion data of the human subject, the potential fall event identified based on one or more rules;
evaluate first samples of the motion data with a machine learning model, to produce a first determination that a fall has occurred during the potential fall event, wherein the first samples correspond to the sensor measurements collected in the subject time period, and wherein use of the machine learning model is invoked in response to the potential fall event identified using the one or more rules;
evaluate second samples of the motion data with a deep learning neural network, to produce a second determination that a fall has occurred during the potential fall event, wherein use of the deep learning neural network is invoked after the use of the machine learning model, in response to the first determination produced from the machine learning model indicating that the fall has occurred, wherein the second samples correspond to the sensor measurements collected in the subject time period and in the earlier time period;
classify the potential fall event as a fall condition for the human subject, based on the first determination produced from the machine learning model and the second determination produced from the deep learning neural network both indicating that the fall has occurred; and
output data to indicate the fall condition for the human subject.

### Claim 21 (depends on 20)

The device-readable storage medium of claim 20, wherein the motion data is provided from a tri-axial accelerometer, wherein the motion data is acceleration data which is downsampled, and wherein the motion data is collected for a predefined amount of time.

### Claim 22 (depends on 20)

The device-readable storage medium of claim 20, wherein the one or more rules relate to acceleration, velocity, or variability of motion, wherein the one or more rules are applied to the motion data to identify the potential fall event.

### Claim 23 (depends on 20)

The device-readable storage medium of claim 20, wherein the machine learning model is a decision tree ensemble model provided from a plurality of decision trees, and wherein the decision tree ensemble model is optimized during training with Bayesian optimization of parameter tuning and feature selection for the plurality of decision trees.

### Claim 24 (depends on 20)

The device-readable storage medium of claim 20, wherein the deep learning neural network is a gated recurrent unit (GRU) model, wherein the GRU model is optimized during training using one or more of: data augmentation, down-sampling of data, variation of a number of layers of the GRU model, or adjusting weights used in training of the GRU model.

### Claim 25 (depends on 20)

The device-readable storage medium of claim 20, the instructions further to cause the electronic device to perform operations that:
evaluate one or more samples of additional motion data with the deep learning neural network, after evaluation of the motion data with the deep learning neural network, wherein the additional motion data includes sensor measurements collected in a later time period after the subject time period;
wherein a counter is maintained to track consecutive fall determinations produced from the deep learning neural network, and wherein classification of the potential fall event as the fall condition for the human subject is further based on the counter exceeding a threshold.

### Claim 26 (depends on 20)

The device-readable storage medium of claim 20, the instructions further to cause the electronic device to perform operations that:
perform fall verification of the potential fall event, before indication of the fall condition for the human subject, wherein the fall verification occurs based on a lack of activity from the human subject within a period of time after the potential fall event.

### Claim 27 (depends on 20)

The device-readable storage medium of claim 20, the instructions further to cause the electronic device to perform operations that:
transmit a communication to a device external to the human subject, the communication including the data to indicate the fall condition for the human subject.

### Claim 28 (depends on 20)

The device-readable storage medium of claim 20, wherein the first determination includes a first probability that a fall has occurred, wherein the second determination includes a second probability that a fall has occurred, wherein use of the deep learning neural network is invoked in response to the first probability exceeding a first probability threshold that a fall has occurred, and wherein the potential fall event is classified as a fall condition for the human subject in response to the second probability exceeding a second probability threshold that a fall has occurred.