# US 12,603,079 B2 · Providing a repository of audio files having pronunciations for text strings to provide to a speech synthesizer > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,603,079 B2 - Document ID: US12603079B2 - Status: Granted - Kind: B2 - Application number: 18/151,953 - Filed: 2023-01-09 - Published: 2024-07-11 - Granted: 2026-04-14 - Earliest claimed: 2023-01-09 - Assignees: International Business Machines Corporation - Applicants: International Business Machines Corporation - Inventors: Jun Su; Yang Liang; Terry James Hoffman; Su Liu - Web page: https://patentgrep.com/patent/US12603079B2 ## CPC - G10L 13/027: Concept to speech synthesisers; Generation of natural phrases from machine-based concepts (generation of parameters for speech synthesis out of text G10L13/08) - G10L 13/08: Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING ## Abstract Provided are a computer program product, system, and method for providing a repository of audio files having pronunciations for text strings to provide to a speech synthesizer. The repository has data structures for text strings in documents. A data structure for a text string indicates at least one attribute of a presentation of the text string in the document and at least one audio file providing at least one audio pronunciation of the text string. A search text string and a search attribute are received from the speech synthesizer. A determination is made of a data structure in the repository including a text string and an attribute matching the search text string and the search attribute, respectively. An audio file, indicated in the determined data structure, is returned to the speech synthesizer to output for the search text string in a document being processed by the speech synthesizer. ## Claims ### Claim 1 A computer program product for providing audio pronunciations to a speech synthesizer to use to convert text to speech in a document, the computer program product comprising a computer readable storage medium having computer readable program code embodied therein that is executable to perform operations, the operations comprising: providing data structures in a repository for text strings in documents, wherein a data structure for a text string in a document indicates at least one attribute of a presentation of the text string in the document and at least one audio file providing at least one audio pronunciation of the text string; receiving, from the speech synthesizer, a search text string and a search attribute; determining a data structure in the repository including a text string and an attribute matching the search text string and the search attribute, respectively; and returning an audio file, indicated in the determined data structure, to the speech synthesizer to output for the search text string in a document being processed by the speech synthesizer. ### Claim 2 (depends on 1) The computer program product of claim 1, wherein the data structures in the repository have a plurality of attributes for presentations of the text strings comprising a category and language in which the text string is presented, wherein the search attribute comprises a plurality of search attributes comprising a search category and a search language, wherein the determined data structure includes a language and category matching the search category and the search language, respectively. ### Claim 3 (depends on 1) The computer program product of claim 1, wherein there are a plurality of the data structures having a same text string and different attributes for different categories and languages. ### Claim 4 (depends on 1) The computer program product of claim 1, wherein the text strings comprise one of an acronym and abbreviation included in the document. ### Claim 5 (depends on 1) The computer program product of claim 1, wherein at least one of the data structures indicates a plurality of audio files providing different pronunciations of a text string for an attribute, and, for each audio file of the indicated plurality of audio files, includes a priority ranking of the indicated plurality of audio files with respect to other of the indicated plurality of audio files, wherein the returned audio file comprises a highest ranked audio file of the indicated plurality of audio files. ### Claim 6 (depends on 1) The computer program product of claim 1, wherein at least one of the data structures indicates a plurality of audio files providing different pronunciations for a text string and includes a count for each of the indicated plurality of audio files indicated in the at least one of the data structures used to rank the indicated plurality of audio files, wherein the operations further comprise: receiving an audio file for a text string and attribute; determining a matching data structure in the repository having the text string and the attribute for the received audio file; determining whether the matching data structure indicates an audio file matching the received audio file; incrementing a count for the audio file, indicated in the matching data structure, in response to determining that the matching data structure indicates the audio file matching the received audio file; and indicating the received audio file in the matching data structure and setting a count in the matching data structure for the received audio file to indicate one instance in response to determining the matching data structure does not have an audio file matching the received audio file. ### Claim 7 (depends on 1) The computer program product of claim 1, wherein the operations further comprise: receiving an audio file for a text string and attribute; determining whether the repository includes a matching data structure indicating the received audio file and the text string and the attribute for the received audio file; adding indication of the received audio file to the matching data structure, determined in the repository, to provide a pronunciation for the text string for the received audio file in response to determining that the repository includes the matching data structure; and adding a data structure to the repository for the received audio file including the text string and the attribute for the received audio file in response to determining that the repository does not include a matching data structure. ### Claim 8 (depends on 1) The computer program product of claim 1, wherein the operations further comprise: processing audio files associated with text strings on a network site; determining an attribute of the text strings for the processed audio files on the network site; and creating data structures in the repository for the processed audio files on the network site, wherein a data structure created for a processed audio file on the network site includes the text string associated with the audio file, the determined attribute of the text string, and access to the audio file. ### Claim 9 (depends on 1) The computer program product of claim 1, wherein the operations further comprise: processing synchronized captions in a transcript of an audio presentation to determine text strings in the synchronized captions comprising acronyms and abbreviations in the synchronized captions; determining audio segments in the audio presentation for the determined text strings in the synchronized captions; determining an attribute of a context of the audio presentation; and creating data structures in the repository for the audio segments, wherein a data structure created for an audio segment of the audio segments includes the text string associated with the audio segment, the determined attribute, and access information for the audio file. ### Claim 10 (depends on 1) The computer program product of claim 1, wherein the operations further comprise: receiving an annotation from a user as input to the speech synthesizer with a user provided audio file for the speech synthesizer to output when processing a specified text string in the document; and creating a data structure in the repository for the user provided audio file including the text string in the annotation and access information for the user provided audio file indicated in the annotation. ### Claim 11 A system for providing audio pronunciations to a speech synthesizer to use to convert text to speech in a document, comprising: a processor; and a computer readable storage medium having computer readable program code embodied therein that when executed by the processor performs operations, the operations comprising: providing data structures in a repository for text strings in documents, wherein a data structure for a text string in a document indicates at least one attribute of a presentation of the text string in the document and at least one audio file providing at least one audio pronunciation of the text string; receiving, from the speech synthesizer, a search text string and a search attribute; determining a data structure in the repository including a text string and an attribute matching the search text string and the search attribute, respectively; and returning an audio file, indicated in the determined data structure, to the speech synthesizer to output for the search text string in a document being processed by the speech synthesizer. ### Claim 12 (depends on 11) The system of claim 11, wherein the data structures in the repository have a plurality of attributes comprising a category and language in which the text string is presented, wherein the search attribute comprises a plurality of search attributes comprising a search category and a search language, wherein the determined data structure includes a language and category matching the search category and the search language, respectively. ### Claim 13 (depends on 11) The system of claim 11, wherein at least one of the data structures indicates a plurality of audio files providing different pronunciations of a text string for an attribute, and, for each audio file of the indicated plurality of audio files, includes a priority ranking of the indicated plurality of audio files with respect to other of the indicated plurality of audio files, wherein the returned audio file comprises a highest ranked audio file of the indicated plurality of audio files. ### Claim 14 (depends on 11) The system of claim 11, wherein at least one of the data structures indicates a plurality of audio files providing different pronunciations for a text string and includes a count for each of the indicated plurality of audio files indicated in the at least one of the data structures used to rank the indicated plurality of audio files, wherein the operations further comprise: receiving an audio file for a text string and attribute; determining a matching data structure in the repository having the text string and the attribute for the received audio file; determining whether the matching data structure indicates an audio file matching the received audio file; incrementing a count for the audio file, indicated in the matching data structure, in response to determining that the matching data structure indicates the audio file matching the received audio file; and indicating the received audio file in the matching data structure and setting a count in the matching data structure for the received audio file to indicate one instance in response to determining the matching data structure does not have an audio file matching the received audio file. ### Claim 15 (depends on 11) The system of claim 11, wherein the operations further comprise: receiving an audio file for a text string and attribute; determining whether the repository includes a matching data structure indicating the received audio file and the text string and the attribute for the received audio file; adding indication of the received audio file to the matching data structure, determined in the repository, to provide a pronunciation for the text string for the received audio file in response to determining that the repository includes the matching data structure; and adding indication of the received audio file to the matching data structure, determined in the repository, to provide a pronunciation for the text string for the received audio file in response to determining that the repository includes the matching data structure; and adding a data structure to the repository for the received audio file including the text string and the attribute for the received audio file in response to determining that the repository does not include a matching data structure. ### Claim 16 A method for providing audio pronunciations to a speech synthesizer to use to convert text to speech in a document, comprising: providing data structures in a repository for text strings in documents, wherein a data structure for a text string in a document indicates at least one attribute of a presentation of the text string in the document and at least one audio file providing at least one audio pronunciation of the text string; receiving, from the speech synthesizer, a search text string and a search attribute; determining a data structure in the repository including a text string and an attribute matching the search text string and the search attribute, respectively; and returning an audio file, indicated in the determined data structure, to the speech synthesizer to output for the search text string in a document being processed by the speech synthesizer. ### Claim 17 (depends on 16) The method of claim 16, wherein the data structures in the repository have a plurality of attributes comprising a category and language in which the text string is presented, wherein the search attribute comprises a plurality of search attributes comprising a search category and a search language, wherein the determined data structure includes a language and category matching the search category and the search language, respectively. ### Claim 18 (depends on 16) The method of claim 16, wherein at least one of the data structures indicates a plurality of audio files providing different pronunciations of a text string for an attribute, and, for each audio file of the indicated plurality of audio files, includes a priority ranking of the indicated plurality of audio files with respect to other of the indicated plurality of audio files, wherein the returned audio file comprises a highest ranked audio file of the indicated plurality of audio files. ### Claim 19 (depends on 16) The method of claim 16, wherein at least one of the data structures indicates a plurality of audio files providing different pronunciations for a text string and includes a count for each of the indicated plurality of audio files indicated in the at least one of the data structures used to rank the indicated plurality of audio files, further comprising: receiving an audio file for a text string and attribute; determining a matching data structure in the repository having the text string and the attribute for the received audio file; determining whether the matching data structure indicates an audio file matching the received audio file; incrementing a count for the received audio file, indicated in the matching data structure, in response to determining that the matching data structure indicates the received audio file matching the received audio file; and indicating the received audio file in the matching data structure and setting a count in the matching data structure for the received audio file to indicate one instance in response to determining the matching data structure does not have an audio file matching the received audio file. ### Claim 20 (depends on 16) The method of claim 16, further comprising: receiving an audio file for a text string and attribute; determining whether the repository includes a matching data structure indicating the received audio file and the text string and the attribute for the received audio file; adding indication of the received audio file to the matching data structure, determined in the repository, to provide a pronunciation for the text string for the received audio file in response to determining that the repository includes the matching data structure; and adding indication of the received audio file to the matching data structure, determined in the repository, to provide a pronunciation for the text string for the received audio file in response to determining that the repository includes the matching data structure; and adding a data structure to the repository for the received audio file including the text string and the attribute for the received audio file in response to determining that the repository does not include a matching data structure.