# US 2021/0280167 A1 · Text to speech prompt tuning by example

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0280167 A1
- Document ID: US20210280167A1
- Status: Published application
- Kind: A1
- Application number: 16/808,914
- Filed: 2020-03-04
- Published: 2021-09-09
- Earliest claimed: 2020-03-04
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: Maria E. Smith; Radek Kazbunda; Michael Alan Picheny; Raul Fernandez
- Web page: https://patentgrep.com/patent/US20210280167A1

## CPC

- G10L 13/10: Prosody rules derived from text; Stress or intonation
- G10L 13/027: Concept to speech synthesisers; Generation of natural phrases from machine-based concepts (generation of parameters for speech synthesis out of text G10L13/08)
- G10L 13/0335: {Pitch control}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

According to one embodiment, a method, computer system, and computer program product for customizing the rendering of a synthesized speech prompt is provided. The present invention may include extracting prosodic information from a received audio recording of a prompt by parsing the text corresponding with the prompt and generating phonetic units, aligning the phonetic units with the audio recording, and calculating, based on the alignment, prosodic values for the phonetic units. The invention may further include adapting the prosodic values to match a text-to-speech voice in use, and then synthesizing speech for the prompt based upon the adapted prosodic information.

## Claims

### Claim 1

A processor-implemented method for customizing the rendering of a synthesized speech prompt, the method comprising:
extracting a plurality of prosodic information from a received audio recording of a prompt; and
synthesizing speech for the prompt based upon the plurality of prosodic information.

### Claim 2 (depends on 1)

The method of claim 1, further comprising: receiving a plurality of text of the prompt.

### Claim 3 (depends on 2)

The method of claim 2, further comprising:
identifying at least one subset of the prompt as dynamic and at least one subset of the prompt as fixed.

### Claim 4 (depends on 3)

The method of claim 3, wherein extracting a plurality of prosodic information is performed only on the subset of the audio recording that corresponds to the fixed subset of the plurality of text.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
associating the prosodic information with the prompt by means of a unique customization identification.

### Claim 6 (depends on 5)

The method of claim 5, wherein the unique identification further comprises a context of the received audio recording.

### Claim 7 (depends on 1)

The method of claim 1, further comprising:
adapting the prosodic information to match a text-to-speech voice.

### Claim 8

A computer system for customizing the rendering of a synthesized speech prompt, the computer system comprising:
one or more processors, one or more computer-readable memories, one or more computer-readable tangible storage medium, and program instructions stored on at least one of the one or more tangible storage medium for execution by at least one of the one or more processors via at least one of the one or more memories, wherein the computer system is capable of performing a method comprising:
extracting a plurality of prosodic information from a received audio recording of a prompt; and
synthesizing speech for the prompt based upon the plurality of prosodic information.

### Claim 9 (depends on 8)

The computer system of claim 8, further comprising:
receiving a plurality of text of the prompt.

### Claim 10 (depends on 9)

The computer system of claim 9, further comprising:
identifying at least one subset of the prompt as dynamic and at least one subset of the prompt as fixed.

### Claim 11 (depends on 10)

The computer system of claim 10, wherein extracting a plurality of prosodic information is performed only on the subset of the audio recording that corresponds to the fixed subset of the plurality of text.

### Claim 12 (depends on 8)

The computer system of claim 8, further comprising:
associating the prosodic information with the prompt by means of a unique customization identification.

### Claim 13 (depends on 12)

The computer system of claim 12, wherein the unique identification further comprises a context of the received audio recording.

### Claim 14 (depends on 8)

The computer system of claim 8, further comprising:
adapting the prosodic information to match a text-to-speech voice.

### Claim 15

A computer program product for customizing the rendering of a synthesized speech prompt, the computer program product comprising:
one or more computer-readable tangible storage medium and program instructions stored on at least one of the one or more tangible storage medium, the program instructions executable by a processor to cause the processor to perform a method comprising:
extracting a plurality of prosodic information from a received audio recording of a prompt; and synthesizing speech for the prompt based upon the plurality of prosodic information.

### Claim 16 (depends on 15)

The computer program product of claim 15, further comprising:
receiving a plurality of text of the prompt.

### Claim 17 (depends on 16)

The computer program product of claim 16, further comprising:
identifying at least one subset of the prompt as dynamic and at least one subset of the prompt as fixed.

### Claim 18 (depends on 17)

The computer program product of claim 17, wherein extracting a plurality of prosodic information is performed only on the subset of the audio recording that corresponds to the fixed subset of the plurality of text.

### Claim 19 (depends on 15)

The computer program product of claim 15, further comprising:
associating the prosodic information with the prompt by means of a unique customization identification.

### Claim 20 (depends on 19)

The computer program product of claim 19, wherein the unique identification further comprises a context of the received audio recording.

### Claim 21 (depends on 15)

The computer program product of claim 15, further comprising:
adapting the prosodic information to match a text-to-speech voice.

### Claim 22

A method for synthesizing speech for a customized prompt, the method comprising:
extracting stored prosodic information for the customized prompt corresponding with a received customization identification;
adapting the prosodic information to match a text-to-speech voice; and
synthesizing speech for the prompt based on the extracted prosodic information.

### Claim 23

A method for extracting prosodic information from an audio recording of a prompt, the method comprising:
parsing the plurality of received text corresponding with the prompt into one or more phonetic units;
aligning the phonetic units with the audio recording; and
calculating, based on the alignment, one or more prosodic values for at least one of the plurality of phonetic units.

### Claim 24 (depends on 23)

The method of claim 23, further comprising paralinguistic detection.

### Claim 25 (depends on 23)

The method of claim 23, wherein the one or more prosodic values enumerate one or more prosodic qualities of a phonetic unit selected from a list consisting of:
a duration, a starting pitch, an ending pitch, a volume, and an additional speech feature.
