# US 2016/0260429 A1 · System and method for automated speech recognition

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2016/0260429 A1
- Document ID: US20160260429A1
- Status: Published application
- Kind: A1
- Application number: 15/029,510
- Filed: 2014-10-14
- Published: 2016-09-08
- Earliest claimed: 2013-10-14
- Assignees: The Penn State Research Foundation
- Applicants: The Penn State Research Foundation
- Inventors: Dezhe Jin; Phillip Baker Schafer
- Web page: https://patentgrep.com/patent/US20160260429A1

## CPC

- G10L 15/20: Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech (G10L21/02 takes precedence)
- G10L 15/02: Feature extraction for speech recognition; Selection of recognition unit
- G10L 15/063: {Training}
- G10L 15/08: Speech classification or search
- G10L 15/142: {Hidden Markov Models [HMMs]}
- G10L 25/21: the extracted parameters being power information
- G10L 25/51: for comparison or discrimination
- G10L 25/87: Detection of discrete points within a voice signal
- G10L 2015/027: {Syllables being the recognition units}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

A method of recognizing speech. The method includes the steps of obtaining an auditory signal; processing the auditory signal into a plurality of frequency components; processing the plurality of frequency components using a plurality of feature detectors, each feature detector producing a feature detector response; generating a spike for each instance in which a feature detector response identifies a characteristic auditory feature to produce a test spike pattern for the auditory signal; and comparing the test spike pattern to a plurality of predetermined spike patterns corresponding to a plurality of speech elements to determine whether the auditory signal includes one of the plurality of speech elements.

## Claims

### Claim 1

A method of recognizing speech, comprising the steps of:
obtaining an auditory signal;
processing the auditory signal into a plurality of frequency components;
processing the plurality of frequency components using a plurality of feature detectors, each feature detector producing a feature detector response;
generating a spike for each instance in which a feature detector response identifies a characteristic auditory feature to produce a test spike pattern for the auditory signal; and
comparing the test spike pattern to a plurality of predetermined spike patterns corresponding to a plurality of speech elements to determine whether the auditory signal includes one of the plurality of speech elements.

### Claim 2 (depends on 1)

The method of claim 1, wherein processing the plurality of frequency components using a plurality of feature detectors comprises inputting a plurality of time-delayed copies of each of the plurality of frequency components to the feature detectors.

### Claim 3 (depends on 1)

The method of claim 1, wherein comparing the test spike pattern to a plurality of predetermined spike patterns comprises
determining a longest common subsequence between the spike test pattern and each of the plurality of predetermined spike patterns, and
identifying the speech element based on the longest common subsequence.

### Claim 4 (depends on 3)

The method of claim 3, wherein identifying the speech element based on the longest common subsequence further comprises calculating a similarity score.

### Claim 5 (depends on 1)

The method of claim 1, wherein each of the plurality of feature detectors comprises a weighting function for the plurality of frequency components.

### Claim 6 (depends on 1)

The method of claim 1, wherein processing the auditory signal into a plurality of frequency components comprises processing the auditory signal into a plurality of frequency components using a plurality of gammatone filters or calculating a spectrogram by computing a short-time Fourier transform of the auditory signal.

### Claim 7 (depends on 6)

The method of claim 6, wherein the gammatone filters have center frequencies from about 100 Hz to about 4000 Hz.

### Claim 8 (depends on 1)

The method of claim 1, wherein the auditory signal speech element comprises a spoken word.

### Claim 9 (depends on 1)

The method of claim 1, wherein each of the plurality of predetermined spike patterns comprises a plurality of feature detector response spikes in an ordered sequence.

### Claim 10 (depends on 1)

The method of claim 1, wherein the plurality of feature detectors comprises a first feature detector and wherein the predetermined spike pattern is generated by a training procedure comprising
obtaining auditory signal positive training data and auditory signal negative training data,
processing the auditory signal positive training data into a plurality of positive training frequency components,
processing the auditory signal negative training data into a plurality of negative training frequency components,
providing the positive training frequency components and the negative training frequency components as inputs to the first feature detector,
weighting the inputs to the first feature detector using a weighting vector to produce a first feature detector response, and
adjusting the weighting vector such that the first feature detector response produces a spike in response to the positive training frequency components and does not produce a spike in response to the negative training frequency components.

### Claim 11 (depends on 10)

The method of claim 10, wherein the auditory signal positive training data comprises less than 100 ms of an auditory signal.

### Claim 12 (depends on 10)

The method of claim 10, wherein the auditory signal negative training data comprises an auditory signal comprising at least one spoken word.

### Claim 13 (depends on 10)

The method of claim 10, wherein the auditory signal positive training data comprises clean speech data.

### Claim 14 (depends on 1)

The method of claim 1, wherein comparing the test spike pattern to a plurality of predetermined spike patterns corresponding to a plurality of speech elements comprises comparing the test spike pattern to a plurality of predetermined spike patterns corresponding to a plurality of speech elements using a hidden Markov model.

### Claim 15 (depends on 1)

The method of claim 1, wherein generating a spike for each instance in which a feature detector response identifies a characteristic auditory feature comprises generating a spike for each instance in which a feature detector response exceeds a predetermined threshold.

### Claim 16 (depends on 1)

The method of claim 1, wherein obtaining an auditory signal comprises identifying a syllable of speech within the auditory signal.

### Claim 17 (depends on 16)

The method of claim 16, wherein identifying a syllable of speech within the auditory signal comprises
identifying a plurality of peaks of acoustic energy of the auditory signal, and
identifying a plurality of minima between the plurality of peaks, wherein a portion of the auditory signal between adjacent minima comprises a syllable of speech.

### Claim 18 (depends on 17)

The method of claim 17, wherein identifying a plurality of peaks of acoustic energy of the auditory signal comprises at least one of smoothing and normalizing the auditory signal.

### Claim 19 (depends on 17)

The method of claim 17, wherein identifying a plurality of peaks of acoustic energy of the auditory signal comprises conditioning the auditory signal, wherein conditioning is selected from the group consisting of: compression, smoothing, normalization, thresholding, false-peak elimination, fitting, and start/end pause priors.

### Claim 20

A system for recognizing speech, comprising:
an acquisition module to obtain an auditory signal; and
a controller in operative communication with the acquisition module, the controller being configured to
obtain an auditory signal from the acquisition module,
process the auditory signal into a plurality of frequency components,
process the plurality of frequency components using a plurality of feature detectors, each feature detector producing a feature detector response,
generate a spike for each instance in which a feature detector response identifies a characteristic auditory feature to produce a test spike pattern for the auditory signal, and
compare the test spike pattern to a plurality of predetermined spike patterns corresponding to a plurality of speech elements to determine whether the auditory signal includes one of the plurality of speech elements.

### Claim 21 (depends on 20)

The system of claim 20, wherein the controller, in order to process the plurality of frequency components using a plurality of feature detectors, is further configured to input a plurality of time-delayed copies of each of the plurality of frequency components to the feature detectors.

### Claim 22 (depends on 20)

The system of claim 20, wherein the controller, in order to compare the test spike pattern to a plurality of predetermined spike patterns, is further configured to
determine a longest common subsequence between the spike test pattern and each of the plurality of predetermined spike patterns, and
identify the speech element based on the longest common subsequence.

### Claim 23 (depends on 22)

The system of claim 22, wherein the controller, in order to identify the speech element based on the longest common subsequence, is further configured to calculate a similarity score.

### Claim 24 (depends on 20)

The system of claim 20, wherein each of the plurality of feature detectors comprises a weighting function for the plurality of frequency components.

### Claim 25 (depends on 20)

The system of claim 20, wherein the controller, in order to process the auditory signal into a plurality of frequency components, is further configured to process the auditory signal into a plurality of frequency components using a plurality of gammatone filters or to calculate a spectrogram by computing a short-time Fourier transform of the auditory signal.

### Claim 26 (depends on 25)

The system of claim 25, wherein the gammatone filters have center frequencies from about 100 Hz to about 4000 Hz.

### Claim 27 (depends on 20)

The system of claim 20, wherein the auditory signal speech element comprises a spoken word.

### Claim 28 (depends on 20)

The system of claim 20, wherein each of the plurality of predetermined spike patterns comprises a plurality of feature detector response spikes in an ordered sequence.

### Claim 29 (depends on 20)

The system of claim 20, wherein the plurality of feature detectors comprises a first feature detector and wherein the controller, in order to generate the predetermined spike pattern, is further configured to
obtain auditory signal positive training data and auditory signal negative training data,
process the auditory signal positive training data into a plurality of positive training frequency components,
process the auditory signal negative training data into a plurality of negative training frequency components,
provide the positive training frequency components and the negative training frequency components as inputs to the first feature detector,
weight the inputs to the first feature detector using a weighting vector to produce a first feature detector response, and
adjust the weighting vector such that the first feature detector response produces a spike in response to the positive training frequency components and does not produce a spike in response to the negative training frequency components.

### Claim 30 (depends on 29)

The system of claim 29, wherein the auditory signal positive training data comprises less than 100 ms of an auditory signal.

### Claim 31 (depends on 29)

The system of claim 29, wherein the auditory signal negative training data comprises an auditory signal comprising at least one spoken word.

### Claim 32 (depends on 29)

The system of claim 29, wherein the auditory signal positive training data comprises clean speech data.

### Claim 33 (depends on 20)

The system of claim 20, wherein the controller, in order to compare the test spike pattern to a plurality of predetermined spike patterns corresponding to a plurality of speech elements, is further configured to compare the test spike pattern to a plurality of predetermined spike patterns corresponding to a plurality of speech elements using a hidden Markov model.

### Claim 34 (depends on 20)

The system of claim 20, wherein the controller, in order to generate a spike for each instance in which a feature detector response identifies a characteristic auditory feature, is further configured to generate a spike for each instance in which a feature detector response exceeds a predetermined threshold.

### Claim 35

A method of recognizing speech, comprising the steps of:
obtaining an auditory signal comprising a plurality of syllables of speech;
identifying a syllable within the auditory signal, the syllable comprising a portion of the auditory signal;
processing the syllable into a plurality of frequency components;
processing the plurality of frequency components using a plurality of feature detectors, each feature detector producing a feature detector response;
generating a spike for each instance in which a feature detector response identifies a characteristic auditory feature to produce a test spike pattern for the auditory signal; and
comparing the test spike pattern to a plurality of predetermined spike patterns corresponding to a plurality of speech elements to determine whether the syllable corresponds to one of the plurality of speech elements.

### Claim 36 (depends on 35)

The method of claim 35, wherein identifying a syllable within the auditory signal comprises
identifying a plurality of peaks of acoustic energy of the auditory signal, and
identifying a plurality of minima between the plurality of peaks, wherein a portion of the auditory signal between adjacent minima comprises the syllable.

### Claim 37 (depends on 36)

The method of claim 36, wherein identifying a plurality of peaks of acoustic energy of the auditory signal comprises at least one of smoothing and normalizing the auditory signal.

### Claim 38 (depends on 36)

The method of claim 36, wherein identifying a plurality of peaks of acoustic energy of the auditory signal comprises conditioning the auditory signal, wherein conditioning is selected from the group consisting of: compression, smoothing, normalization, thresholding, false-peak elimination, fitting, and start/end pause priors.

### Claim 39 (depends on 35)

The method of claim 35, wherein one of the plurality of speech elements comprises a first syllable and a second syllable, and wherein the syllable identified within the auditory signal corresponds to the first syllable or the second syllable.
