System-effected methods for analyzing, predicting, and/or modifying acoustic units of human utterances for use in speech synthesis and recognition

A method for categorically mapping the relationship of at least one text unit in a sequence of text to at least one corresponding prosodic phonetic unit, to at least one linguistic feature category in the sequence of text, and to at least one speech utterance represented in a synthesized speech signal, the method comprising:

• (a) identifying, and optionally modifying, acoustic data representing the at least one speech utterance, to provide the synthesized speech signal;
• (b) identifying, and optionally modifying, the acoustic data representing the at least one utterance to provide the at least one speech utterance with an expressive prosody determined according to prosodic rules;
• and
• (c) identifying acoustic unit feature vectors for each of the at least one prosodic phonetic units, each acoustic unit feature vector comprising a bundle of feature values selected according to proximity to a statistical mean of the values of acoustic unit candidates available for matching with the respective prosodic phonetic unit and, optionally, for acoustic continuity with at least one adjacent acoustic feature vector.

A method of deriving an acoustic path comprising a sequence of desired acoustic units extending through unconstrained acoustic space, the acoustic path being useful for synthesizing speech from text with a desired style of speech prosody by concatenating the sequence of desired acoustic units, the method comprising:

• (a) providing a database of acoustic units wherein each acoustic unit is identified according to a prosodic phonetic unit name and at least one additional linguistic feature;
• and wherein each acoustic unit has been analyzed according to phase-state metrics so that pitch, energy, and spectral wave data can be modified simultaneously at one or more instants in time;
• (b) mapping each acoustic unit to prosodic phonetic unit categorizations and additional linguistic categorizations enabling the acoustic unit to be specified and/or altered to provide one or more acoustic units for incorporation into expressively synthesized speech according to prosodic rules;
• (c) calculating weighted absolute and/or relative acoustic values for a set of candidate acoustic units to match each desired acoustic unit, one candidate set per desired acoustic unit, matching being in terms of linguistic features for the corresponding mapped prosodic phonetic unit or a substitute for the corresponding mapped prosodic phonetic unit;
• (d) calculating an acoustic path through n-dimensional acoustic space to be sequenced as an utterance of synthesized speech, the acoustic path being defined by the weighted average values for each candidate set of acoustic units;
• (e) selecting and modifying, as needed, a sequence of acoustic units, or sub-units, for the synthesized speech according to the differences between the weighted acoustic values for a candidate acoustic unit, or sub-unit, and the weighted acoustic values of a point on the calculated acoustic path.