# US 10,478,111 B2 · Systems for speech-based assessment of a patient's state-of-mind

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,478,111 B2
- Document ID: US10478111B2
- Status: Granted
- Kind: B2
- Application number: 15/505,577
- Filed: 2015-08-05
- Published: 2018-08-02
- Granted: 2019-11-19
- Earliest claimed: 2014-08-22
- Assignees: SRI International
- Applicants: SRI International
- Inventors: Bruce Knoth; Dimitra Vergyri; Elizabeth Shriberg; Vikramjit Mitra; Mitchell McLaren; Andreas Kathol; Colleen Richey; Martin Graciarena
- Web page: https://patentgrep.com/patent/US10478111B2

## CPC

- A61B 5/165: {Evaluating the state of mind, e.g. depression, anxiety}
- G06N 20/00: Machine learning
- G10L 15/1807: {using prosody or stress}
- G10L 15/1822: {Parsing for meaning understanding}
- G10L 17/26: Recognition of special voice characteristics, e.g. for use in lie detectors; Recognition of animal voices
- G10L 25/63: for estimating an emotional state
- G10L 25/66: for extracting parameters related to health condition (detecting or measuring for diagnostic purposes A61B5/00)
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

A computer-implemented method can include a speech collection module collecting a speech pattern from a patient, a speech feature computation module computing at least one speech feature from the collected speech pattern, a mental health determination module determining a state-of-mind of the patient based at least in part on the at least one computed speech feature, and an output module providing an indication of a diagnosis with regard to a possibility that the patient is suffering from a certain condition such as depression or Post-Traumatic Stress Disorder (PTSD).

## Claims

### Claim 1

A computer-implemented method of improving output of a speech-based state-of-mind detection system, the method comprising:
receiving, using the speech-based state-of-mind detection system, a speech pattern collected from a patient;
determining, using the speech-based state-of-mind detection system, at least one feature of the collected speech pattern;
determining, using the speech-based state-of-mind detection system, a presence or absence of a state-of-mind of the patient using a first distribution of a non-lexical speech feature computed over a first time interval in which speech of the patient is collected during a session and a second distribution of the non-lexical speech feature computed over a second time interval in which speech of the patient is collected during the session to indicate a change in the patient from a first state-of-mind to a second state-state-of-mind as a result of the session;
wherein the state-of-mind is defined by a clinical protocol; and
outputting, by the speech-based state-of-mind detection system, the presence or absence of the state-of-mind to the patient, a clinical expert, or both the patient and the clinical expert.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, wherein the determined state-of-mind indicates a probability that the patient is suffering from at least one of:
depression, suicidal tendencies, Post-Traumatic Stress Disorder (PTSD), a concussion, bipolar disorder, anxiety disorder, mild Traumatic Brain Injury (mTBI), and schizophrenia.

### Claim 3 (depends on 1)

The computer-implemented method of claim 1, further comprising using at least the at least one non-lexical speech feature to build a predictor of the patient's state-of-mind.

### Claim 4 (depends on 3)

The computer-implemented method of claim 3, wherein the using is performed on a remote server.

### Claim 5 (depends on 1)

The computer-implemented method of claim 1, wherein the at least one non-lexical speech feature includes linguistic features and at least one of: prosodic features and spectral features.

### Claim 6 (depends on 1)

The computer-implemented method of claim 1, wherein the at least one non-lexical speech feature corresponds to a pitch of the patient's speech.

### Claim 7 (depends on 1)

The computer-implemented method of claim 1, wherein the at least one non-lexical speech feature corresponds to a harmonicity of the patient's speech.

### Claim 8 (depends on 1)

The computer-implemented method of claim 1, further comprising comparing the determined state-of-mind to an empirical threshold.

### Claim 9 (depends on 1)

The computer-implemented method of claim 1, further comprising: prompting the patient to produce the speech pattern; producing the speech pattern responsive to the prompting; and receiving the produced speech pattern.

### Claim 10 (depends on 9)

The computer-implemented method of claim 9, further comprising analyzing the quality of the received speech pattern.

### Claim 11 (depends on 10)

The computer-implemented method of claim 10, wherein analyzing the quality of the received speech pattern comprises determining whether the analyzed quality of the received speech pattern meets a particular threshold.

### Claim 12 (depends on 11)

The computer-implemented method of claim 11, further comprising causing the prompting, producing, receiving, and analyzing to repeat until the analyzed quality of the received speech pattern meets the particular threshold.

### Claim 13 (depends on 9)

The computer-implemented method of claim 9, wherein prompting the patient to produce the speech pattern comprises presenting to the patient a plurality of open-ended questions.

### Claim 14

A system for improving output of an automated speech-based state-of-mind detection process, the system comprising at least one processor configured to:
receive, using the speech-based state-of-mind detection process, a speech pattern collected from a patient;
compute, from the collected speech pattern, using the speech-based state-of-mind detection process, at least one non-lexical speech feature that indicates a condition of equipment used to collect the speech pattern or a physical environment in which the speech pattern was collected;
determine a presence or absence of a state-of-mind of the patient using the at least one computed non-lexical speech feature that indicates a condition of equipment used to collect the speech pattern or a physical environment in which the speech pattern was collected,
wherein the state-of-mind is defined by a clinical protocol,
wherein the presence or absence of the state-of-mind is using at least one model that models speech patterns of healthy speakers and models speech patterns of speakers that have been positively diagnosed with the state-of-mind; and
outputting, by the speech-based state-of-mind detection process, an indication of the presence or absence of the state-of-mind to the patient, a clinical expert, or both the patient and the clinical expert.

### Claim 15 (depends on 14)

The system of claim 14, wherein the at least one processor is configured to:
store speech assessment information; and
develop an algorithm configured to be used by the mental health determination module in determining the state-of-mind of the patient.

### Claim 16 (depends on 14)

The system of claim 14, wherein the at least one processor is configured to interact with the mental health determination module over a network.

### Claim 17 (depends on 16)

The system of claim 16, wherein the network is the Internet.

### Claim 18 (depends on 14)

The system of claim 14, wherein the at least one processor is configured to use at least the at least one computed speech feature to build a predictor of the patient's state-of-mind.

### Claim 19 (depends on 14)

The system of claim 14, wherein the at least one processor is a same processor.

### Claim 20 (depends on 14)

The system of claim 14, wherein the determined state-of-mind indicates a probability that the patient is suffering from at least one of: depression, suicidal tendencies, Post-Traumatic Stress Disorder (PTSD), a concussion, bipolar disorder, anxiety disorder, mild Traumatic Brain Injury (mTBI), and schizophrenia.

### Claim 21 (depends on 14)

The system of claim 14, wherein the at least one computed speech feature includes linguistic features and at least one of: prosodic features and spectral features.

### Claim 22 (depends on 14)

The system of claim 14, wherein the at least one speech feature corresponds to a pitch of the patient's speech.

### Claim 23 (depends on 14)

The system of claim 14, wherein the at least one speech feature corresponds to a harmonicity of the patient's speech.

### Claim 24 (depends on 14)

The system of claim 14, wherein the collected speech pattern comprises an answer to a question that was presented to the patient.

### Claim 25 (depends on 14)

The system of claim 14, wherein the collected speech pattern comprises a plurality of answers, each answer corresponding to one of a plurality of questions that were presented to the patient.

### Claim 26 (depends on 14)

The system of claim 14, wherein the at least one processor is configured to check the audio quality of the collected speech pattern.

### Claim 27

A computer-implemented method for improving output of a speech-based state-of-mind detection system, the method comprising:
receiving, by the speech-based state-of-mind detection system, a speech pattern collected from a patient;
computing, by the speech-based state-of-mind detection system, a plurality of speech features from the collected speech pattern;
determining, by the speech-based state-of-mind detection system using the plurality of speech features, a presence or absence of a state-of-mind of the patient;
wherein the state-of-mind is defined by a clinical protocol,
wherein the presence or absence of the state-of-mind is determined using at least one model that models speech patterns of healthy speakers and models speech patterns of speakers that have been positively diagnosed with the state-of-mind,
wherein the plurality of speech features comprises at least two of any of the following: Damped Oscillator Cepstral Coefficients (DOCCs), Normalized Modulation Cepstral Coefficients (NMCCs), Medium Duration Speech Amplitudes (MMeDuSA), Gammatone Cepstral Coefficients (GCCs), DeepTVs, a syllable duration, a speaking rate, a speech rhythmicity, a tilt feature;
outputting, by the speech-based state-of-mind detection system, the determined state-of-mind to the patient or a clinical expert or both the patient and the clinical expert.