# US 11,443,733 B2 · Contextual text-to-speech processing > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,443,733 B2 - Document ID: US11443733B2 - Status: Granted - Kind: B2 - Application number: 16/665,886 - Filed: 2019-10-28 - Published: 2020-05-14 - Granted: 2022-09-13 - Earliest claimed: 2017-03-02 - Assignees: Amazon Technologies, Inc. - Applicants: Amazon Technologies, Inc. - Inventors: Roberto Barra Chicote; Javier Latorre; Adam Franciszek Nadolski; Viacheslav Klimkov; Thomas Edward Merritt - Web page: https://patentgrep.com/patent/US11443733B2 ## CPC - G10L 13/10: Prosody rules derived from text; Stress or intonation - G10L 13/033: Voice editing, e.g. manipulating the voice of the synthesiser - G10L 13/047: Architecture of speech synthesisers - G10L 2013/105: {Duration} - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING ## Abstract A text-to-speech (TTS) system that is capable of considering characteristics of various portions of text data in order to create continuity between segments of synthesized speech. The system can analyze text portions of a work and create feature vectors including data corresponding to characteristics of the individual portions and/or the overall work. A TTS processing component can then consider feature vector(s) from other portions when performing TTS processing on text of a first portion, thus giving the TTS component some intelligence regarding other portions of the work, which can then result in more continuity between synthesized speech segments. ## Claims ### Claim 1 A computer-implemented method comprising: receiving text data including a first text portion, a second text portion and a third text portion, the first text portion representing a first plurality of words, the second text portion representing a second plurality of words, and the third text portion representing a third plurality of words; determining the first text portion and the second text portion correspond to a first contextual section; determining the third text portion corresponds to a second contextual section different from the first contextual section; based at least in part on the first text portion and the second text portion corresponding to the first contextual section, and the third text portion corresponding to the second contextual section, determining to perform text-to-speech (TTS) processing with respect to the first text portion using contextual information from the second text portion rather than the third text portion; processing the first text portion to determine first data corresponding to a representation of the first plurality of words; processing the second text portion to determine second data representing context information corresponding to the second text portion; and performing TTS processing using the first data and the second data to determine audio data corresponding to the first text portion. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, further comprising: determining the first text portion and the second text portion correspond to a first dialogue section; and determining the third text portion corresponds to a second dialogue section, wherein the TTS processing uses the second data in response to the first text portion and the second text portion corresponding to the first dialogue section. ### Claim 3 (depends on 1) The computer-implemented method of claim 1, further comprising: determining that the first text portion corresponds to a first paragraph; and determining that the second text portion corresponds to a second paragraph contiguous with the first paragraph, wherein the TTS processing uses the second data in response to the second text portion corresponding to the second paragraph contiguous with the first paragraph. ### Claim 4 (depends on 3) The computer-implemented method of claim 3, further comprising: determining an indication corresponding to a paragraph break, wherein performing the TTS processing further uses the indication. ### Claim 5 (depends on 1) The computer-implemented method of claim 1, further comprising: determining that a total of the first plurality of words and the second plurality of words exceeds a threshold number of words, wherein the TTS processing uses the second data in response to the total exceeding the threshold number of words. ### Claim 6 (depends on 1) The computer-implemented method of claim 1, further comprising: determining that the first text portion corresponds to a chapter heading for a first chapter; determining that the second text portion corresponds to a text within the first chapter; and determining an indication corresponding to a chapter heading pause, wherein performing the TTS processing further uses the indication. ### Claim 7 (depends on 1) The computer-implemented method of claim 1, further comprising: receiving second text data including a fourth text portion representing a fourth plurality of words; determining the first text portion, the second text portion, and the fourth text portion correspond to a first contextual section; and processing the second text data to determine third data representing second context information corresponding to the fourth text portion, wherein performing the TTS processing further uses the third data. ### Claim 8 (depends on 1) The computer-implemented method of claim 1, further comprising: determining that a first total of the first plurality of words and the second plurality of words does not exceed a threshold number of words; receiving second text data including a fourth text portion representing a fourth plurality of words; determining that a second total of the first plurality of words, the second plurality of words, and the fourth plurality of words exceeds the threshold number of words; and processing the second text data to determine third data representing second context information corresponding to the fourth text portion, wherein the TTS processing further uses the third data in response to the second total exceeding the threshold number of words. ### Claim 9 (depends on 1) The computer-implemented method of claim 1, further comprising: processing the second text portion to determine third data corresponding to a representation of the second plurality of words; processing the first text portion to determine fourth data representing context information corresponding to the first text portion; and performing TTS processing using the third data and the fourth data to determine second audio data corresponding to the second text portion. ### Claim 10 A system, comprising: at least one processor; at least one memory comprising instructions that, when executed by the at least one processor, cause the system to: receive text data including a first text portion, a second text portion and a third text portion, the first text portion representing a first plurality of words, the second text portion representing a second plurality of words, and the third text portion representing a third plurality of words; determine the first text portion and the second text portion correspond to a first contextual section; determine the third text portion corresponds to a second contextual section different than the first contextual section; based at least in part on the first text portion and the second text portion corresponding to the first contextual section, and the third text portion corresponding to the second contextual section, determine to perform text-to-speech (TTS) processing with respect to the first text portion using contextual information from the second text portion rather than the third text portion; process the first text portion to determine first data corresponding to a representation of the first plurality of words; process the second text portion to determine second data representing context information corresponding to the second text portion; and perform TTS processing using the first data and the second data to determine audio data corresponding to the first text portion. ### Claim 11 (depends on 10) The system of claim 10, wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to: determine the first text portion and the second text portion correspond to a first dialogue section; and determine the third text portion corresponds to a second dialogue section, wherein the TTS processing uses the second data in response to the first text portion and the second text portion corresponding to the first dialogue section. ### Claim 12 (depends on 10) The system of claim 10, wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to: determine that the first text portion corresponds to a first paragraph; and determine that the second text portion corresponds to a second paragraph contiguous with the first paragraph, wherein the TTS processing uses the second data in response to the second text portion corresponding to the second paragraph contiguous with the first paragraph. ### Claim 13 (depends on 12) The system of claim 12, wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to: determine an indication corresponding to a paragraph break, wherein the TTS processing further uses the indication. ### Claim 14 (depends on 10) The system of claim 10, wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to: determine that a total of the first plurality of words and the second plurality of words exceeds a threshold number of words, wherein the TTS processing uses the second data in response to the total exceeding the threshold number of words. ### Claim 15 (depends on 10) The system of claim 10, wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to: determine that the first text portion corresponds to a chapter heading for a first chapter; determine that the second text portion corresponds to a text within the first chapter; and determine an indication corresponding to a chapter heading pause, wherein the TTS processing further uses the indication. ### Claim 16 (depends on 10) The system of claim 10, wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to: receive second text data including a fourth text portion representing a fourth plurality of words; determine the first text portion, the second text portion, and the fourth text portion correspond to a first contextual section; and process the second text data to determine third data representing second context information corresponding to the fourth text portion, wherein the TTS processing further uses the third data. ### Claim 17 (depends on 10) The system of claim 10, wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to: determine that a first total of the first plurality of words and the second plurality of words does not exceed a threshold number of words; receive second text data including a fourth text portion representing a fourth plurality of words; determine that a second total of the first plurality of words, the second plurality of words, and the fourth plurality of words exceeds the threshold number of words; and process the second text data to determine third data representing second context information corresponding to the fourth text portion, wherein the TTS processing further uses the third data in response to the second total exceeding the threshold number of words. ### Claim 18 (depends on 10) The system of claim 10, wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to: process the second text portion to determine third data corresponding to a representation of the second plurality of words; process the first text portion to determine fourth data representing context information corresponding to the first text portion; and perform TTS processing using the third data and the fourth data to determine second audio data corresponding to the second text portion.