# US 12,118,980 B2 · Artificial intelligence-based text-to-speech system and method > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,118,980 B2 - Document ID: US12118980B2 - Status: Granted - Kind: B2 - Application number: 18/346,657 - Filed: 2023-07-03 - Published: 2023-11-02 - Granted: 2024-10-15 - Earliest claimed: 2017-05-18 - Assignees: Telepathy Labs, Inc. - Applicants: Telepathy Labs, Inc. - Inventors: Martin Reber; Vijeta Avijeet - Web page: https://patentgrep.com/patent/US12118980B2 ## CPC - G10L 13/08: Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination - G06F 18/10: Pre-processing; Data cleansing - G06F 18/2135: based on approximation criteria, e.g. principal component analysis - G06F 18/217: {Validation; Performance evaluation; Active pattern learning techniques} - G06N 3/02: Neural networks - G06N 3/042: Knowledge-based neural networks; Logical representations of neural networks - G06N 3/0495: Quantised networks; Sparse networks; Compressed networks - G06N 3/0499: Feedforward networks - G06N 3/08: Learning methods - G06N 3/09: Supervised learning - G06N 5/02: Knowledge representation; Symbolic representation - G10L 13/04: Details of speech synthesis systems, e.g. synthesiser structure or memory management - G10L 19/00: Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis (in musical instruments G10H) - G10L 25/30: using neural networks - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract A technique improves training and speech quality of a text-to-speech (TTS) system having an artificial intelligence, such as a neural network. The TTS system is organized as a front-end subsystem and a back-end subsystem. The front-end subsystem is configured to provide analysis and conversion of text into input vectors, each having at least a base frequency, f0, a phenome duration, and a phoneme sequence that is processed by a signal generation unit of the back-end subsystem. The signal generation unit includes the neural network interacting with a pre-existing knowledgebase of phenomes to generate audible speech from the input vectors. The technique applies an error signal from the neural network to correct imperfections of the pre-existing knowledgebase of phenomes to generate audible speech signals. A back-end training system is configured to train the signal generation unit by applying psychoacoustic principles to improve quality of the generated audible speech signal. ## Claims ### Claim 1 A text-to-speech (TTS) system including one or more processors and one or more memories configured to perform operations for converting text into a corrected speech signal comprising: training a neural network based upon, at least in part, data of previously generated speech in a pre-existing knowledgebase of phonemes, wherein the previously generated speech has an inaccuracy; generating a lossy representation of at least a portion of the data for use in the training; and applying lossy representation of at least the portion of the data to the previously generated speech for correcting the inaccuracy of the previously generated speech in the pre-existing knowledgebase of phonemes. ### Claim 2 (depends on 1) The TTS system of claim 1 wherein the lossy representation reduces a representation of a phoneme in the pre-existing knowledgebase of phonemes resulting in a lossy representation of the phoneme where the inaccuracy represents inaudible errors. ### Claim 3 (depends on 1) The TTS system of claim 1 wherein generating the lossy representation includes limiting derivations of a principal component analysis for at least the portion of the data. ### Claim 4 (depends on 1) The TTS system of claim 1 wherein the lossy representation is a domain-to-frequency domain transformation of at least the portion of the data. ### Claim 5 (depends on 1) The TTS system of claim 1 wherein applying the lossy representation includes correcting voiced phonemes of the pre-existing knowledgebase of phonemes using principal component analysis. ### Claim 6 (depends on 1) The TTS system of claim 1 wherein applying the lossy representation includes correcting unvoiced phonemes of the pre-existing knowledgebase of phonemes using noise band/energy band thresholding. ### Claim 7 (depends on 1) The TTS system of claim 1 wherein applying the lossy representation includes combining a limited number of frequency bands with specified band widths. ### Claim 8 A method of processing text-to-speech (TTS) comprising: training a neural network based upon, at least in part, data of previously generated speech in a pre-existing knowledgebase of phonemes, wherein the previously generated speech has an inaccuracy; generating a lossy representation of at least a portion of the data for use in the training; and applying lossy representation of at least the portion of the data to the previously generated speech for correcting the inaccuracy of the previously generated speech in the pre-existing knowledgebase of phonemes. ### Claim 9 (depends on 8) The method of claim 8 wherein the lossy representation reduces a representation of a phoneme in the pre-existing knowledgebase of phonemes resulting in a lossy representation of the phoneme where the inaccuracy represents inaudible errors. ### Claim 10 (depends on 8) The method of claim 8 wherein generating the lossy representation includes limiting derivations of a principal component analysis for at least the portion of the data. ### Claim 11 (depends on 8) The method of claim 8 wherein the lossy representation is a domain-to-frequency domain transformation of at least the portion of the data. ### Claim 12 (depends on 8) The method of claim 8 wherein applying the lossy representation includes correcting voiced phonemes of the pre-existing knowledgebase of phonemes using principal component analysis. ### Claim 13 (depends on 8) The method of claim 8 wherein applying the lossy representation includes correcting unvoiced phonemes of the pre-existing knowledgebase of phonemes using noise band/energy band thresholding. ### Claim 14 (depends on 8) The method of claim 8 wherein applying the lossy representation includes combining a limited number of frequency bands with specified band widths. ### Claim 15 A non-transitory computer-readable medium having program instructions which, when executed across one or more processors, causes at least a portion of the one or more processors to perform operations comprising: training a neural network based upon, at least in part, data of previously generated speech in a pre-existing knowledgebase of phonemes, wherein the previously generated speech has an inaccuracy; generating a lossy representation of at least a portion of the data for use in the training; and applying lossy representation of at least the portion of the data to the previously generated speech for correcting the inaccuracy of the previously generated speech in the pre-existing knowledgebase of phonemes. ### Claim 16 (depends on 15) The non-transitory computer-readable medium of claim 15 wherein the lossy representation reduces a representation of a phoneme in the pre-existing knowledgebase of phonemes resulting in a lossy representation of the phoneme where the inaccuracy represents inaudible errors. ### Claim 17 (depends on 15) The non-transitory computer-readable medium of claim 15 wherein generating the lossy representation includes limiting derivations of a principal component analysis for at least the portion of the data. ### Claim 18 (depends on 15) The non-transitory computer-readable medium of claim 15 wherein applying the lossy representation includes correcting voiced phonemes of the pre-existing knowledgebase of phonemes using principal component analysis. ### Claim 19 (depends on 15) The non-transitory computer-readable medium of claim 15 wherein applying the lossy representation includes correcting unvoiced phonemes of the pre-existing knowledgebase of phonemes using noise band/energy band thresholding. ### Claim 20 (depends on 15) The non-transitory computer-readable medium of claim 15 wherein applying the lossy representation includes combining a limited number of frequency bands with specified band widths.