# US 2014/0195242 A1 · Prosody Generation Using Syllable-Centered Polynomial Representation of Pitch Contours > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2014/0195242 A1 - Document ID: US20140195242A1 - Status: Published application - Kind: A1 - Application number: 14/216,611 - Filed: 2014-03-17 - Published: 2014-07-10 - Earliest claimed: 2012-12-03 - Inventors: Chengjun Julian Chen - Web page: https://patentgrep.com/patent/US20140195242A1 ## CPC - G10L 13/02: Methods for producing synthetic speech; Speech synthesisers - G10L 13/0335: {Pitch control} - G10L 13/10: Prosody rules derived from text; Stress or intonation - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING ## Abstract The present invention discloses a parametrical representation of prosody based on polynomial expansion coefficients of the pitch contour near the center of each syllable. The said syllable pitch expansion coefficients are generated from a recorded speech database, read from a number of sentences by a reference speaker. By correlating the stress level and context information of each syllable in the text with the polynomial expansion coefficients of the corresponding spoken syllable, a correlation database is formed. To generate prosody for an input text, stress level and context information of each syllable in the text is identified. The prosody is generated by using the said correlation database to find the best set of pitch parameters for each syllable. By adding to global pitch contours and using interpolation formulas, complete pitch contour for the input text is generated. Duration and intensity profile are generated using a similar procedure. ## Claims ### Claim 1 A method for building databases for prosody generation in speech synthesis using one or more processors comprising: A) compile a text corpus of sentences containing all the prosody phenomena of interest; B) for each phrase in each said sentence, identify the phrase type; C) segment each sentence into syllables, identify the property and context information of each said syllable; D) read the sentences by a reference speaker to make a recording of voice signals with simultaneous electroglottograph signals if an electroglottograph instrument is available; E) segment the voice signals and electroglottograph signals of each sentence into syllables, each said syllable is aligned with a syllable in the text; F) identify the voiced section in each syllable of the voice recording; G) calculate pitch values in the said voiced section; H) generate a polynomial expansion of the pitch contour of each said voiced section in each syllable by least-squares fitting, comprising the use of Gegenbauer polynomials, which at least have a constant term representing the average pitch of the said syllable; I) for all phrases of a given type, generate a polynomial expansion of the values of said average pitch of all syllables in the said phrases using least-squares fitting, to generate an average global pitch contour of the given phrase type; J) form a set of syllable pitch parameters for each said syllable by subtracting the value of the global pitch profile at that point from the value of the average pitch of the said syllable together with the rest of polynomial expansion coefficients for the said syllable; K) correlate the syllable pitch parameters with the property and context information of the said syllable from an analysis of the text to form a database of syllable pitch parameters; L) correlate the intensity and duration parameters of a syllable to the property and context information of the said syllable from an analysis of the text to form a database of intensity and duration. ### Claim 2 (depends on 1) The pitch values in claim 1 is a linear function of the logarithm of the pitch period, preferably in MIDI unit. ### Claim 3 (depends on 1) The property and context information of the said syllable in claim 1 comprises the stress level of the said syllable in a word, the emphasis level, part of speech, grammatical identity of the said word in the phrase, and the similar information of neighboring syllables and words. ### Claim 4 (depends on 1) For tone languages such as Mandarin Chinese, the property and context information in claim 1 comprises the tone and stress level of the said syllable in a word, the emphasis level, part of speech, grammatical identity of the said word in the phrase, and the similar information of neighboring syllables and words. ### Claim 5 (depends on 1) The type of phrase in claim 1 comprises declarative, interrogative, exclamatory, or intermediate phrase. ### Claim 6 (depends on 1) A method for generating prosody in speech synthesis from an input sentence using the said databases in claim 1 comprising: A) for each phrase in the said input sentence, identify the phrase type; B) segment each sentence into syllables, identify the property and context information of each said syllable; C) based on the said phrase type, retrieving a global phrase pitch profile from the global pitch profiles database for each said phrase; D) finding the syllable pitch parameters for each said syllable using the property and context information of each said syllable and the database of syllable pitch parameters; E) for each said syllable, adding the pitch value in the global pitch contour at the time of the said syllable to the constant term of the said syllable pitch parameters; F) calculating pitch values for the entire sentence using polynomial interpolation; G) finding the intensity and duration parameters for each said syllable using the property and context information of each said syllable and the database of intensity and duration parameters; H) output the said pitch contour and said intensity and duration parameters for the entire sentence as prosody parameters for speech synthesis. ### Claim 7 (depends on 6) The pitch values in claim 6 is a linear function of the logarithm of the pitch period, preferably in MIDI unit. ### Claim 8 (depends on 6) The property and context information in claim 6 comprises the stress level of the said syllable in a word, the emphasis level, part of speech, grammatical identity of the said word in the phrase, and the similar information of neighboring syllables and words. ### Claim 9 (depends on 6) For tone languages such as Mandarin Chinese, the property and context information in claim 6 comprises the tone and stress level of the said syllable in a word, the emphasis level, part of speech, grammatical identity of the said word in the phrase, and the similar information of neighboring syllables and words. ### Claim 10 (depends on 6) The type of phrase in claim 6 comprises declarative, interrogative, exclamatory, or intermediate phrase. ### Claim 11 A system for building databases for prosody generation in speech synthesis using one or more processors comprising: A) compile a text corpus of sentences containing all the prosody phenomena of interest; B) for each phrase in each said sentence, identify the phrase type; C) segment each sentence into syllables, identify the property and context information of each said syllable; D) read the sentences by a reference speaker to make a recording of voice signals with simultaneous electroglottograph signals if an electroglottograph instrument is available; E) segment the voice signals and electroglottograph signals of each sentence into syllables, each said syllable is aligned with a syllable in the text; F) identify the voiced section in each syllable of the voice recording; G) calculate pitch values in the said voiced section; H) generate a polynomial expansion of the pitch contour of each said voiced section in each syllable by least-squares fitting, comprising the use of Gegenbauer polynomials, which at least have a constant term representing the average pitch of the said syllable; I) for all phrases of a given type, generate a polynomial expansion of the values of said average pitch of all syllables in the said phrases using least-squares fitting, to generate an average global pitch contour of the given phrase type; J) form a set of syllable pitch parameters for each said syllable by subtracting the value of the global pitch profile at that point from the value of the average pitch of the said syllable together with the rest of polynomial expansion coefficients for the said syllable; K) correlate the syllable pitch parameters with the property and context information of the said syllable from an analysis of the text to form a database of syllable pitch parameters; L) correlate the intensity and duration parameters of a syllable to the property and context information of the said syllable from an analysis of the text to form a database of intensity and duration. ### Claim 12 (depends on 11) The pitch values in claim 11 is a linear function of the logarithm of the pitch period, preferably in MIDI unit. ### Claim 13 (depends on 11) The property and context information in claim 11 comprises the stress level of the said syllable in a word, the emphasis level, part of speech, grammatical identity of the said word in the phrase, and the similar information of neighboring syllables and words. ### Claim 14 (depends on 11) For tone languages such as Mandarin Chinese, the property and context information in claim 11 comprises the tone and the stress level of the said syllable in a word, the emphasis level, part of speech, grammatical identity of the said word in the phrase, and the similar information of neighboring syllables and words. ### Claim 15 (depends on 11) The type of phrase in claim 11 comprises declarative, interrogative, exclamatory, or intermediate phrase; ### Claim 16 (depends on 11) A system for generating prosody in speech synthesis from an input sentence using the said databases in claim 11 comprising: A) for each phrase in the said sentence, identify the phrase type; B) segment each sentence into syllables, identify the property and context information of each said syllable; C) based on the said phrase type, selecting a global phrase pitch profile from the global pitch profiles database for each said phrase; D) finding the syllable pitch parameters for each said syllable using the property and context information of each said syllable and the database of syllable pitch parameters; E) for each said syllable, adding the pitch value in the global pitch contour at the time of the said syllable to the constant term of the said syllable pitch parameters; F) calculating pitch values for the entire sentence using polynomial interpolation; G) finding the intensity and duration parameters for each said syllable using the property and context information of each said syllable and the database of intensity and duration parameters; H) output the said pitch contour and said intensity and duration parameters for the entire sentence as prosody parameters for speech synthesis. ### Claim 17 (depends on 16) The pitch values in claim 16 is a linear function of the logarithm of the pitch period, preferably in MIDI unit. ### Claim 18 (depends on 16) The property and context information in claim 16 comprises the stress level of the said syllable in a word, the emphasis level, part of speech, grammatical identity of the said word in the phrase, and the similar information of neighboring syllables and words. ### Claim 19 (depends on 16) For tone languages such as Mandarin Chinese, the property and context information in claim 16 comprises the tone and the stress level of the said syllable in a word, the emphasis level, part of speech, grammatical identity of the said word in the phrase, and the similar information of neighboring syllables and words. ### Claim 20 (depends on 16) The type of phrase in claim 16 comprises declarative, interrogative, exclamatory, or intermediate phrase.