US 12,609,131 B2Granted

Tracking articulatory and prosodic development in children

AssigneesArizona Board of Regents on behalf of Arizona State University, Wisconsin Alumni Research Foundation, Linus Health, Inc.

InventorsVisar Berisha, Julie Liss, Katherine Hustad, Tristan Mahr, Kan Kawabata

Life of this patent

Estimating the term…

patentgrep

Loading patent…

Abstract

Systems, devices, and methods for tracking articulatory and prosodic development in children are disclosed. Human speech in a given language can be divided into phonemes, which are a sound or group of sounds perceived by speakers of the language to have a common linguistic function (e.g., consonant sounds, vowel sounds). In an exemplary aspect, a normative model can be generated for production characteristics of each phoneme in a given language using a database of normative speech samples. One or more speech samples of a human subject can be analyzed to identify the phonemes used by the human subject and measured against the normative model. Based on this analysis, a normed score is generated of the articulation accuracy, duration, rhythm, volume, and/or other production characteristics for each phoneme of the speech sample of the human subject.

Claims21

Dependent claims

A method for processing speech, the method comprising:

generating from production data of normative speech samples of a plurality of mature speakers, a plurality of statistical models defining production characteristics associated with a respective target phoneme of a plurality of target phonemes of a target language sample;
receiving, from a human subject having an associated age, a speech sample including a plurality of phonemes;
extracting acoustic features from the speech sample of the human subject including a plurality of spectro-temporal patterns;
aligning the speech sample with the target language sample to identify, for each target phoneme of the plurality of target phonemes, a plurality of instances of the target phoneme in the speech sample and to analyze acoustics of each instance of the phoneme relative to a respective target phoneme of the plurality of target phonemes of the target language sample;
assessing, probabilistically, for articulatory precision and other characteristics of each of the plurality of target phonemes elicited by the human subject in the speech sample separately relative to a respective target phoneme of the plurality of target phonemes of the target language sample, including, for each instance of a given phoneme:
projecting the acoustic features of each frame of the speech sample within the instance on a respective statistical model of the plurality of statistical models, and
computing, for the given phoneme, a normed score including phoneme log-likelihood ratio (PLLR) values from the acoustic features of the speech sample as projected on the respective statistical model of the plurality of statistical models, including:
determining, for each instance of the given phoneme, a ratio of:
a probability of a correct pronunciation of a given phoneme of the speech sample relative to a respective target phoneme, to
a maximum probability of a correct pronunciation of other phonemes of the speech sample relative to respective target phonemes, and
applying a logarithmic transformation to the ratio to obtain a PLLR value for each instance of the given phoneme;
for each given phoneme of the plurality of target phonemes, aggregating the PLLR values over all instances of the given phoneme in the speech sample to compute an average articulatory precision metric for the given phoneme;
and
for each given phoneme of the plurality of target phonemes, comparing the average articulatory precision metric with a normative distribution of articulatory precision development by age and phoneme derived from PLLR values of a population of typical speakers to compute a normed score that quantifies a distance of the human subject from mature production for the given phoneme at the associated age.

11 more elements

A method for assessing speech, the method comprising:

accessing a first speech sample of a human subject;
spectrally sampling the first speech sample;
analyzing the spectrally sampled first speech sample to separate the first speech sample into a plurality of instances of a plurality of phonemes;
for each given phoneme of the plurality of phonemes, analyzing acoustic features of the plurality of phonemes separately, including
comparing each of the plurality of instances of the given phoneme with a statistical model of the given phoneme to produce an estimate of correct pronunciation, the estimate of correct pronunciation for each instance comprising a phoneme log-likelihood ratio (PLLR) value obtained by determining, for the instance, a ratio of a probability of a correct pronunciation of the given phoneme of the speech sample relative to a respective target phoneme to a maximum probability of a correct pronunciation of other phonemes of the speech sample relative to respective target phonemes, and applying a logarithmic transformation to the ratio;
and
generating,
for each given phoneme of the plurality of phonemes, a normed score by aggregating the PLLR values over the plurality of instances of the given phoneme in the first speech sample to compute an average articulatory precision metric for the given phoneme, and comparing the average articulatory precision metric with a normative distribution of articulatory precision development by age and phoneme derived from PLLR values of a population of typical speakers to compute the normed score as a value that quantifies a distance of the human subject from mature production for the given phoneme at the associated age.

5 more elements

A speech processing system comprising:

a database comprising normative acoustic data for a plurality of phonemes;
and
audio processing circuitry coupled to the database and configured to:
produce a model of each of the plurality of phonemes based on the normative acoustic data;
receive a speech sample comprising a plurality of instances of each of the plurality of phonemes from a human subject having an associated age;
and
for each given phoneme, separately:
spectrally analyze the speech sample to identify the plurality of instances of the given phoneme;
and
compare the plurality of instances of the given phoneme to the model of the given phoneme to compute, for each instance of the given phoneme, a phoneme log-likelihood ratio (PLLR) value defining a log ratio of a probability of a correct pronunciation of the given phoneme of the speech sample relative to a respective target phoneme to a maximum probability of a correct pronunciation of other phonemes of the speech sample relative to respective target phonemes, to aggregate the PLLR values over the plurality of instances of the given phoneme in the speech sample to compute an average articulatory precision metric for the given phoneme, and to compare the average articulatory precision metric with a normative distribution of articulatory precision development by age and phoneme derived from PLLR values of a population of typical speakers to produce a normed score for the given phoneme that quantifies a distance of the human subject from mature production for the given phoneme at the associated age.

7 more elements

Dependencies are inferred from the claim text.

Facts

Application: 17/764,054Filed 28 Sep 2020 · us provisional application
Priority: 27 Sep 2019
Published: US 2022/0415308 A129 Dec 2022
Granted: 21 Apr 2026
Term ends: Not estimatedThe source records don’t support an estimate.

Classification10 codes

G10L 15/26
Speech recognition: speech to text systems
G10L 25/51
Specially adapted for particular use: for comparison or discrimination
G10L 15/02
Speech recognition: feature extraction for speech recognition; Selection of recognition unit
G10L 15/187
Using context dependencies, e.g. language models: phonemic context, e.g. pronunciation rules, phonotactical constraints or phoneme n-grams