# US 2018/0322894 A1 · Selecting speech features for building models for detecting medical conditions

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2018/0322894 A1
- Document ID: US20180322894A1
- Status: Published application
- Kind: A1
- Application number: 15/973,498
- Filed: 2018-05-07
- Published: 2018-11-08
- Earliest claimed: 2017-05-05
- Assignees: Canary Speech, LLC
- Applicants: Canary Speech, LLC
- Inventors: JANGWON KIM; NAMHEE KWON; HENRY O'CONNELL; PHILLIP WALSTAD; KEVIN SHENGBIN YANG
- Web page: https://patentgrep.com/patent/US20180322894A1

## CPC

- G16H 80/00: ICT specially adapted for facilitating communication between medical practitioners or patients, e.g. for collaborative diagnosis, therapy or health monitoring
- A61B 5/1123: {Discriminating type of movement, e.g. walking or running (A61B5/1116, A61B5/112 take precedence)}
- A61B 5/4088: {Diagnosing of monitoring cognitive diseases, e.g. Alzheimer, prion diseases or dementia}
- A61B 5/4803: {Speech analysis specially adapted for diagnostic purposes}
- A61B 5/7267: {involving training the classification device}
- G06N 3/0499: Feedforward networks
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G06N 20/10: using kernel methods, e.g. support vector machines [SVM]
- G10L 25/66: for extracting parameters related to health condition (detecting or measuring for diagnostic purposes A61B5/00)
- G16H 10/20: for electronic clinical trials or questionnaires
- G16H 40/67: for remote operation
- G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems
- G16H 50/50: for simulation or modelling of medical disorders
- G06F 2111/10: Numerical modelling
- G10L 15/02: Feature extraction for speech recognition; Selection of recognition unit
- G10L 15/063: {Training}
- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A mathematical model may be trained to diagnose a medical condition of a person by processing acoustic features and language features of speech of the person. The performance of the mathematical model may be improved by appropriately selecting the features to be used with the mathematical model. Features may be selected by computing a feature selection score for each acoustic feature and each language feature, and then selecting features using the scores, such as by selecting features with the highest scores. In some implementations, stability determinations may be computed for each feature and features may be selected using both the feature selection scores and the stability determinations. A mathematical model may then be trained using the selected features and deployed. In some implementations, prompts may be selected using computed prompt selection scores, and the deployed mathematical model may be used with the selected prompts.

## Claims

### Claim 1

A system for training a mathematical model for detecting a medical condition, the system comprising at least one computer configured to:
obtain a training corpus comprising speech data items, wherein each speech data item is labelled with a diagnosis value;
obtain speech recognition results for each speech data item using automatic speech recognition, wherein the speech recognition results for a speech data item comprise a transcription of the speech data item;
compute a plurality of acoustic features for each speech data item in the training corpus, wherein the plurality of acoustic features is computed from the speech data item and wherein computation of the plurality of acoustic features does not use the speech recognition results of the speech data item;
compute a plurality of language features for each speech data item in the training corpus by processing the speech recognition results;
compute a feature selection score for each feature of the plurality of acoustic features and each feature of the plurality of language features, wherein:
the feature selection score for a feature indicates a usefulness of the feature for detecting the medical condition, and
the feature selection score is computed using, for each speech data item, a value of the feature and the diagnosis value corresponding to the speech data item;
select a plurality of features from the plurality of acoustic features and the plurality of language features using the feature selection scores;
train the mathematical model for detecting the medical condition using the selected plurality of features for each speech data item of the training corpus;
deploy a computer program product or computer service for detecting the medical condition using the mathematical model;
present, by the computer program product or computer service, a prompt to a person;
receive, by the computer program product or computer service, a speech data item corresponding to speech of a person in response to the prompt;
compute a medical diagnosis score by processing the received speech data item using the mathematical model; and
display, by the computer program product or computer service, one or more of the medical diagnosis score or a medical diagnosis based on the medical diagnosis score.

### Claim 2

(canceled)

### Claim 3 (depends on 1)

The system of claim 1, wherein each speech data item of the training corpus corresponds to a prompt of a plurality of prompts, the plurality of prompts comprising the presented prompt, and wherein the at least one computer is configured to:
compute a medical diagnosis score for each speech data item of the training corpus by processing the speech data items with the mathematical model;
compute a prompt selection score for each prompt of the plurality of prompts using the medical diagnosis scores;
select a subset of prompts from the plurality of prompts using the prompt selection scores, the subset of prompts comprising the presented prompt;
deploy the computer program product or computer service for detecting the medical condition using the mathematical model and the subset of prompts;
receive a speech data item corresponding to speech of the person for each prompt of the subset of prompts; and
compute the medical diagnosis score for the person by processing the received speech data items using the mathematical model.

### Claim 4 (depends on 1)

The system of claim 1, wherein the at least one computer is configured to:
divide the training corpus into a plurality of folds; and
compute a statistic for each feature and each fold of the plurality of folds.

### Claim 5 (depends on 4)

The system of claim 4, wherein the at least one computer is configured to:
compute a stability determination for each feature of the plurality of acoustic features and the plurality of language features using the statistics for each feature and each fold of the plurality of folds; and
select the plurality of features using the stability determinations.

### Claim 6 (depends on 1)

The system of claim 1, wherein the mathematical model comprises a neural network or a support vector machine.

### Claim 7 (depends on 1)

The system of claim 1, wherein the plurality of acoustic features comprises at least one of spectral features, prosodic features, or voice quality features.

### Claim 8

A computer-implemented method for training a mathematical model for detecting a medical condition, the method comprising:
obtaining a training corpus comprising speech data items, wherein each speech data item is labelled with a diagnosis value;
obtaining speech recognition results for each speech data item using automatic speech recognition, wherein the speech recognition results for a speech data item comprise a transcription of the speech data item;
computing a plurality of acoustic features for each speech data item in the training corpus, wherein the plurality of acoustic features is computed from the speech data item and wherein computation of the plurality of acoustic features does not use the speech recognition results of the speech data item;
computing a plurality of language features for each speech data item in the training corpus by processing the speech recognition results;
computing a feature selection score for each feature of the plurality of acoustic features and each feature of the plurality of language features, wherein:
the feature selection score for a feature indicates a usefulness of the feature for detecting the medical condition, and
the feature selection score is computed using, for each speech data item, a value of the feature and the diagnosis value corresponding to the speech data item;
selecting a plurality of features from the plurality of acoustic features and the plurality of language features using the feature selection scores;
training the mathematical model for detecting the medical condition using the selected plurality of features for each speech data item of the training corpus;
deploying a computer program product or computer service for detecting the medical condition using the mathematical model;
presenting, by the computer program product or computer service, a prompt to a person;
receiving, by the computer program product or computer service, a speech data item corresponding to speech of a person in response to the prompt;
computing a medical diagnosis score by processing the received speech data item using the mathematical model; and
displaying, by the computer program product or computer service, one or more of the medical diagnosis score or a medical diagnosis based on the medical diagnosis score.

### Claim 9 (depends on 8)

The computer-implemented method of claim 8, wherein the medical condition is a concussion or Alzheimer's disease.

### Claim 10 (depends on 8)

The computer-implemented method of claim 8, wherein the plurality of language features comprises one or more of a number of pause fillers over a period of time, a number of pause fillers over a number of words, word difficulty, or speaking rate.

### Claim 11 (depends on 8)

The computer-implemented method of claim 8, wherein computing a feature selection score for a feature comprises generating a pair of numbers for each speech data item of the training corpus, and wherein a first number of the pair corresponds to a feature value and a second number of the pair corresponds to a diagnosis value.

### Claim 12 (depends on 8)

The computer-implemented method of claim 8, comprising:
dividing the training corpus into a plurality of folds; and
computing a statistic for each feature and each fold of the plurality of folds.

### Claim 13 (depends on 12)

The computer-implemented method of claim 12, comprising:
computing a stability determination for each feature of the plurality of acoustic features and the plurality of language features using the statistics for each feature and each fold of the plurality of folds; and
selecting the plurality of features using the stability determinations.

### Claim 14 (depends on 8)

The computer-implemented method of claim 8, comprising:
selecting a plurality of prompts using the mathematical model; and
training a second mathematical model using the selected plurality of prompts and the speech data items of the training corpus.

### Claim 15

One or more non-transitory computer-readable media comprising computer executable instructions that, when executed, cause at least one processor to perform actions comprising:
obtaining a training corpus comprising speech data items, wherein each speech data item is labelled with a diagnosis value;
obtaining speech recognition results for each speech data item using automatic speech recognition, wherein the speech recognition results for a speech data item comprise a transcription of the speech data item;
computing a plurality of acoustic features for each speech data item in the training corpus, wherein the plurality of acoustic features is computed from the speech data item and wherein computation of the plurality of acoustic features does not use the speech recognition results of the speech data item;
computing a plurality of language features for each speech data item in the training corpus by processing the speech recognition results;
computing a feature selection score for each feature of the plurality of acoustic features and each feature of the plurality of language features, wherein:
the feature selection score for a feature indicates a usefulness of the feature for detecting a medical condition, and
the feature selection score is computed using, for each speech data item, a value of the feature and the diagnosis value corresponding to the speech data item;
selecting a plurality of features from the plurality of acoustic features and the plurality of language features using the feature selection scores;
training a mathematical model for detecting the medical condition using the selected plurality of features for each speech data item of the training corpus;
deploying a computer program product or computer service for detecting the medical condition using the mathematical model;
presenting, by the computer program product or computer service, a prompt to a person;
receiving, by the computer program product or computer service, a speech data item corresponding to speech of a person in response to the prompt;
computing a medical diagnosis score by processing the received speech data item using the mathematical model; and
displaying, by the computer program product or computer service, one or more of the medical diagnosis score or a medical diagnosis based on the medical diagnosis score.

### Claim 16 (depends on 15)

The one or more non-transitory computer-readable media of claim 15, wherein computing a first acoustic feature of the plurality of acoustic features comprises:
computing a value for each short-time segment of an audio signal to obtain a plurality of values; and
computing the first acoustic feature using the plurality of values.

### Claim 17 (depends on 15)

The one or more non-transitory computer-readable media of claim 15, wherein the feature selection score comprises an adjusted Rand index, adjusted mutual information, an absolute Pearson correlation, or an absolute Spearman correlation.

### Claim 18 (depends on 15)

The one or more non-transitory computer-readable media of claim 15, wherein the actions comprise:
computing a stability determination for each feature of the plurality of acoustic features and the plurality of language features; and
selecting the plurality of features using the stability determinations.

### Claim 19 (depends on 15)

The one or more non-transitory computer-readable media of claim 15, wherein each speech data item of the training corpus corresponds to a prompt of a plurality of prompts, the plurality of prompts comprising the presented prompt, and wherein the actions comprise:
computing a medical diagnosis score for each speech data item of the training corpus by processing the speech data items with the mathematical model;
computing a prompt selection score for each prompt of the plurality of prompts using the medical diagnosis scores;
selecting a subset of prompts from the plurality of prompts using the prompt selection scores, the subset of prompts comprising the presented prompt; and
deploying the computer program product or computer service for detecting the medical condition using the mathematical model and the subset of prompts.

### Claim 20 (depends on 15)

The one or more non-transitory computer-readable media of claim 15, wherein the actions comprise:
obtaining a plurality of non-speech features for each speech data item in the training corpus;
computing a feature selection score for each feature of the plurality of non-speech features;
selecting the plurality of features using the feature selection scores for the plurality of non-speech features.