# US 11,026,613 B2 · System, device and method for remotely monitoring the well-being of a user with a wearable device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,026,613 B2
- Document ID: US11026613B2
- Status: Granted
- Kind: B2
- Application number: 15/555,813
- Filed: 2016-03-09
- Published: 2018-02-15
- Granted: 2021-06-08
- Earliest claimed: 2015-03-09
- Assignees: Koninklijke Philips N.V.
- Applicants: Koninklijke Philips N.V.
- Inventors: John E. Cronin; Steven M. Philbin; Eric R. Wade
- Web page: https://patentgrep.com/patent/US11026613B2

## CPC

- A61B 5/165: {Evaluating the state of mind, e.g. depression, anxiety}
- A61B 5/0022: {Monitoring a patient using a global network, e.g. telephone networks, internet}
- A61B 5/0205: Simultaneously evaluating both cardiovascular conditions and different types of body conditions, e.g. heart and respiratory condition
- A61B 5/021: Measuring pressure in heart or blood vessels
- A61B 5/02405: {Determining heart rate variability}
- A61B 5/4803: {Speech analysis specially adapted for diagnostic purposes}
- A61B 5/4809: {Sleep detection, i.e. determining whether a subject is asleep or not}
- A61B 5/6802: {Sensor mounted on worn items}
- A61B 5/746: {Alarms related to a physiological condition, e.g. details of setting alarm thresholds or avoiding false alarms}
- G10L 25/63: for estimating an emotional state
- G10L 25/78: Detection of presence or absence of voice signals (switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10)
- G10L 25/84: for discriminating voice from noise
- A61B 5/02438: {with portable devices, e.g. worn by the patient}
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

Systems, devices, methods for providing a speech pattern as a metric of well-being system for remotely monitoring the well-being of a patient are disclosed. In one exemplary embodiment, a system can include at least one wearable device that is configured to collect body sensor data and speech pattern data associated with a patient wearing the device and analyze the data to determine if the patient's emotional well-being is compromised. In some exemplary embodiments, the wearable device can be configured to send an alert to at least one caregiver device that indicates the patient's emotional well-being is compromised. The wearable device can also be configured to send recommendations on courses of action to alleviate the condition.

## Claims

### Claim 1

A method of remotely monitoring emotional well-being of a user with a wearable device having a speech sensor and at least one body sensor comprising:
monitoring speech pattern data from the speech sensor;
monitoring body sensor data from at least one body sensor;
comparing the speech pattern data and the body sensor data to an alert database having a plurality of different emotional well-being alerts, each of the different emotional well-being alerts comprising a message tailored to a unique combination of speech pattern data and body sensor data;
selecting, from the plurality of different emotional well-being alerts based on the comparison, an emotional well-being alert; and
sending, by a communication module, the selected alert message to a caregiver;
wherein said monitoring of speech pattern data includes calculating a non-interaction value based on a percent of time that the user has not spoken, wherein the calculating of the non-interaction value is also based on at least one factor for normalizing the percent of time to a baseline percent of time that the user has not spoken, and wherein the calculating of the non-interaction value is also based on at least one factor depending on a type of non-interaction activity occurring during the time not speaking detected by the at least one body sensor of the wearable device, wherein the type of non-interaction activity detected affects the comparison of the speech pattern data and the body sensor data, the emotional well-being alerts including a depression alert that is triggered when the non-interaction value is less than a minimum non-interaction value and a reading from a body sensor is below a predetermined value; and
said sending the selected alert message includes sending a depression alert message when the depression alert is triggered.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the depression alert is triggered when the non-interaction value is less than the minimum non-interaction value and a blood pressure reading from a wearable device blood pressure sensor is less than a minimum blood pressure.

### Claim 3 (depends on 1)

The method according to claim 1, wherein the depression alert is triggered when the non-interaction value is less than the minimum non-interaction value and a pulse reading from a wearable device pulse sensor is less than or equal to a minimum pulse.

### Claim 4 (depends on 1)

The method according to claim 1, wherein the depression alert is triggered when the non-interaction value is less than the minimum non-interaction value and a heart rate variability reading from a wearable device pulse sensor is less than or equal to a minimum heart rate variability.

### Claim 5 (depends on 1)

The method according to claim 1, wherein said calculating the non-interaction value includes:
recording audio data obtained by the speech sensor;
analyzing the audio data to isolate user speech from ambient sound; and
calculating a percent of time that the user has not spoken from the user speech audio.

### Claim 6 (depends on 5)

The method according to claim 5, wherein said analyzing the audio data to isolate user speech from ambient sound includes isolating user speech from sounds associated with television sounds, radio sounds, and speech from people other than the user.

### Claim 7 (depends on 1)

The method according to claim 1, wherein the at least one body sensor includes a movement sensor for sensing when the user moves, and wherein said monitoring speech pattern data includes not monitoring speech pattern data during a predefined night time period unless the movement sensor senses movement indicating the user is awake.

### Claim 8 (depends on 1)

The method according to claim 1, wherein the depression alert is triggered when the non-interaction value is less than the minimum non-interaction value, the reading from the at least one body sensor is below the predetermined value, and a weather value received by the wearable device indicates poor weather.

### Claim 9 (depends on 1)

The method according to claim 1, wherein:
the emotional well-being alerts include a stress alert that is triggered when a reading from at least one body sensor is greater than a maximum value and a speech pattern is detected by the speech sensor; and
said sending the selected alert message includes sending a stress alert message when the stress alert is triggered.

### Claim 10 (depends on 9)

The method according to claim 9, wherein the speech pattern detected by the speech sensor includes detecting user speech indicating the user is speaking and ambient speech indicating another person is speaking.

### Claim 11 (depends on 10)

The method according to claim 10, wherein the speech pattern detected includes detecting a conversation speech pattern when both user speech and ambient speech are detected and analysis of the speech sensor data indicates the user is involved in a conversation, said sending the selected alert message includes sending a stress alert message that the user is engaged in a conversation that is causing stress.

### Claim 12 (depends on 11)

The method according to claim 11, wherein the speech pattern detected includes detecting an argument speech pattern when both user speech and ambient speech are detected and analysis of the speech sensor data indicates the user is involved in an argument, said emotional well-being alerts including a conversation stress alert that is triggered when the conversation speech pattern is detected and the reading from the at least one body sensor exceeds a first maximum value, said emotional well-being alerts further including an argument stress alert that is triggered when the argument speech pattern is detected and the reading from the at least one body sensor exceeds a second maximum value that is lower than the first maximum value.

### Claim 13 (depends on 9)

The method according to claim 9, wherein the stress alert is triggered when a reading from one of a wearable device blood pressure sensor and a wearable device pulse sensor exceed a corresponding maximum value and a speech pattern is detected by the speech sensor.

### Claim 14 (depends on 9)

The method according to claim 9, wherein the reading from the at least one body sensor is at least one of a user heart rate from a wearable device pulse sensor and a user blood pressure from a wearable device blood pressure sensor and the maximum value is at least one of a maximum heart rate and a maximum blood pressure.

### Claim 15

A system for remotely monitoring the emotional well-being of a user, comprising:
at least one wearable device having:
a speech sensor structured to sense speech pattern data, and
one or more body sensors structured to sense body sensor data; and
at least one computing device having at least one data processor and structured to collect and monitor data; and
wherein the at least one wearable device is configured to at least one of (i) collect and monitor speech pattern data from the speech sensor, or (ii) collect and monitor body sensor data from the one or more body sensors, and
wherein the at least one computing device is configured to (i) compare at least one of the speech pattern data or the body sensor data to data from one or more databases having a plurality of different emotional well-being alerts, each of the different emotional well-being alerts comprising a message tailored to a unique combination of speech pattern data and body sensor data, and (ii) determine whether to trigger at least one of the different emotional well-being alerts based on the comparison, and
wherein the at least one wearable device is further structured and configured to at least one of store, display, indicate or send an alert message when at least one of the emotional well-being alerts are triggered; and
wherein the at least one computing device is configured to (i) calculate a non-interaction value based on a percent of time that the user has not spoken, wherein the calculation is further based on at least one factor for normalizing the percent of time to a baseline percent of time that the user has not spoken, and wherein the calculation is further based on at least one factor depending on a type of non-interaction activity occurring during the time not speaking detected by the at least one body sensor of the wearable device, wherein the type of non-interaction activity detected affects the comparison of the speech pattern data and the body sensor data, and at least one of the emotional well-being alerts includes a depression alert that is triggered when the non-interaction value is less than a minimum non-interaction value and a reading from one or more of the body sensors is below a predetermined value, and (ii) send a depression alert message when the depression alert is triggered.

### Claim 16 (depends on 1)

The method according to claim 1, further comprising updating at least one body sensor threshold of the alert database based on a geographic location of the wearable device.