# US 2023/0410787 A1 · Speech processing system with encoder-decoder model and corresponding methods for synthesizing speech containing desired speaker identity and emotional style > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2023/0410787 A1 - Document ID: US20230410787A1 - Status: Published application - Kind: A1 - Application number: 18/314,408 - Filed: 2023-05-09 - Published: 2023-12-21 - Earliest claimed: 2022-05-18 - Assignees: Innocorn Technology Limited - Applicants: Innocorn Technology Limited - Inventors: Kwok Leung LEE - Web page: https://patentgrep.com/patent/US20230410787A1 ## CPC - G06N 3/0455: Auto-encoder networks; Encoder-decoder networks - G10L 13/027: Concept to speech synthesisers; Generation of natural phrases from machine-based concepts (generation of parameters for speech synthesis out of text G10L13/08) - G10L 13/033: Voice editing, e.g. manipulating the voice of the synthesiser - G10L 13/047: Architecture of speech synthesisers - G10L 17/26: Recognition of special voice characteristics, e.g. for use in lie detectors; Recognition of animal voices - G10L 25/30: using neural networks - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING ## Abstract Embodiments of the present application provide a speech processing system, a speech processing method, and a terminal device. The speech processing system is mounted on a terminal device, the speech processing system includes at least one speech engine module, the at least one speech engine module is provided therein with a trained deep learning model, and the trained deep learning model includes an encoder and a decoder, where the encoder is configured to obtain a speaker identity when a to-be-processed speech service is played; and the decoder is configured to process the speaker identity, text information corresponding to the to-be-processed speech service, and an emotional style expected to be output by the to-be-processed speech service to obtain a processing result of the to-be-processed speech service. The present application can enrich a speech output effect output by using a text to speech (TTS) engine. ## Claims ### Claim 1 A speech processing system, wherein the speech processing system is mounted on a terminal device, the speech processing system comprises at least one speech engine module, the at least one speech engine module is provided therein with a trained deep learning model, and the trained deep learning model comprises an encoder and a decoder, wherein the encoder is configured to obtain a speaker identity when a to-be-processed speech service is played; and the decoder is configured to process the speaker identity, text information corresponding to the to-be-processed speech service, and an emotional style expected to be output by the to-be-processed speech service to obtain a processing result of the to-be-processed speech service. ### Claim 2 (depends on 1) The system according to claim 1, wherein the encoder has an N-layer structure, and each layer comprises a first multi-head self-attention layer and a first feedforward neural network; and the decoder has an N-layer structure, and each layer comprises a second multi-head self-attention layer, a second feedforward neural network, and a multi-head attention layer; wherein the multi-head attention layer is configured to execute a multi-head attention mechanism for the speaker identity output by the encoder, and N is an integer greater than 0. ### Claim 3 (depends on 1) The system according to claim 1, wherein the speech processing system further comprises a configuration module and a speech control module that are connected to the at least one speech engine module, wherein the configuration module is configured to receive a playback parameter set by a user for the processing result of the to-be-processed speech service; and the speech control module is configured to play the processing result of the to-be-processed speech service based on the playback parameter. ### Claim 4 (depends on 1) The system according to claim 1, wherein the speaker identity when the to-be-processed speech service is played comprises one hundred people of different ages from all walks of life. And the speaking identity is related to the number of combinations of all the dimensions in the speaker embeddings which describing the voice characteristics of the speaker. ### Claim 5 (depends on 1) The system according to claim 1, wherein the emotional style expected to be output by the to-be-processed speech service comprises one of happiness, sadness, disgust, fear, surprise, and anger. And with a larger dataset with more emotion styles, it can represent more. ### Claim 6 A speech processing method, wherein the speech processing method is applied to a speech processing system, wherein the speech processing system comprises at least one speech engine module, the at least one speech engine module is provided therein with a trained deep learning model, and the trained deep learning model comprises an encoder and a decoder; and the speech processing method comprises: obtaining a to-be-processed speech service; obtaining, by using the encoder, a speaker identity when the to-be-processed speech service is played; and processing, by using the decoder, the speaker identity, text information corresponding to the to-be-processed speech service, and an emotional style expected to be output by the to-be-processed speech service to obtain a processing result of the to-be-processed speech service. ### Claim 7 (depends on 6) The method according to claim 6, wherein the encoder has an N-layer structure, and each layer comprises a first multi-head self-attention layer and a first feedforward neural network; and the decoder has an N-layer structure, and each layer comprises a second multi-head self-attention layer, a second feedforward neural network, and a multi-head attention layer; wherein the multi-head attention layer is configured to execute a multi-head attention mechanism for the speaker identity output by the encoder, and N is an integer greater than 0. ### Claim 8 (depends on 6) The method according to claim 6, wherein the speech processing system further comprises a configuration module and a speech control module that are connected to the at least one speech engine module, and the method further comprises: receiving, by using the configuration module, a playback parameter set by a user for the processing result of the to-be-processed speech service; and playing, by using the speech control module, the processing result of the to-be-processed speech service based on the playback parameter. ### Claim 9 (depends on 6) The method according to claim 6, wherein the speaker identity when the to-be-processed speech service is played comprises one hundred people of different ages from all walks of life. And the speaking identity is related to the number of combinations of all the dimensions in the speaker embeddings which describing the voice characteristics of the speaker. ### Claim 10 (depends on 6) The method according to claim 6, wherein the emotional style expected to be output by the to-be-processed speech service comprises one of happiness, sadness, disgust, fear, surprise, and anger. And with a larger dataset with more emotion styles, it can represent more.