# US 2003/0229496 A1 · Speech synthesis method and apparatus, and dictionary generation method and apparatus

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2003/0229496 A1
- Document ID: US20030229496A1
- Status: Published application
- Kind: A1
- Application number: 10/449,072
- Filed: 2003-06-02
- Published: 2003-12-11
- Earliest claimed: 2002-06-05
- Assignees: Canon Kabushiki Kaisha
- Inventors: Masayuki Yamada; Yasuhiro Komori; Toshiaki Fukada
- Web page: https://patentgrep.com/patent/US20030229496A1

## CPC

- G10L 13/06: Elementary speech units used in speech synthesisers; Concatenation rules
- G10L 13/04: Details of speech synthesis systems, e.g. synthesiser structure or memory management
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

In a speech synthesis process, micro-segments are cut from acquired waveform data and a window function. The obtained micro-segments are re-arranged to implement a desired prosody, and superposed data is generated by superposing the re-arranged micro-segments, so as to obtain synthetic speech waveform data. A spectrum correction filter is formed based on the acquired waveform data. At least one of the waveform data, micro-segments, and superposed data is corrected using the spectrum correction filter. In this way, “blur” of a speech spectrum due to the window function applied to obtain micro-segments is reduced, and speech synthesis with high sound quality is realized.

## Claims

### Claim 1

A speech synthesis method comprising:
an acquisition step of acquiring micro-segments from speech waveform data and a window function;
a re-arrangement step of re-arranging the micro-segments acquired in the acquisition step to change prosody upon synthesis;
a synthesis step of outputting synthetic speech waveform data on the basis of superposed waveform data obtained by superposing the micro-segments re-arranged in the re-arrangement step; and
a correction step of correcting at least one of the speech waveform data, the micro-segments, and the superposed waveform data using a spectrum correction filter formed based on the speech waveform data to be processed in the acquisition step.

### Claim 2 (depends on 1)

The method according to claim 1, further comprising:
a formation step of forming the spectrum correction filter formed on the basis of the speech waveform data to be processed in the acquisition step.

### Claim 3 (depends on 1)

The method according to claim 1, further comprising:
a speech synthesis dictionary which registers formation information for a spectrum correction filter in correspondence with each speech waveform data, and
wherein the correction step includes a step of forming the spectrum correction filter by acquiring formation information corresponding to the speech waveform data to be processed in the acquisition step from the speech synthesis dictionary.

### Claim 4 (depends on 1)

The method according to claim 1, further comprising:
a speech synthesis dictionary which registers spectrum-corrected speech waveform data obtained by applying a spectrum correction filter formed based on each speech waveform data to that speech waveform data, and
wherein the acquisition step includes a step of acquiring the micro-segments from the spectrum-corrected speech waveform data obtained from the speech synthesis dictionary, and the window function.

### Claim 5 (depends on 1)

The method according to claim 1, further comprising:
a formation step of forming the spectrum correction filter formed on the basis of the speech waveform data to be processed in the acquisition step, and separating the spectrum correction filter into a plurality of element filters, and
wherein the correction step includes a step of applying each of the plurality of element filters obtained in the formation step to not less than two out of the speech waveform data, the micro-segments, and the superposed waveform data.

### Claim 6 (depends on 5)

The method according to claim 5, wherein the formation step includes a step of forming three element filters from the spectrum correction filter, and
the correction step includes a step of applying the three element filters to the speech waveform data, the micro-segments, and the superposed waveform data, respectively.

### Claim 7 (depends on 5)

The method according to claim 5, wherein the formation step includes a step of obtaining the plurality of element filters by factorizing a characteristic polynomial that represents the spectrum correction filter to convert the spectrum correction filter into a product of a plurality of element filters.

### Claim 8 (depends on 5)

The method according to claim 5, wherein the formation step includes a step of obtaining the plurality of element filters by approximating the spectrum correction filter by a filter expressed by a polynomial, and factorizing the polynomial to convert the spectrum correction filter into a product of element filters.

### Claim 9 (depends on 1)

The method according to claim 1, further comprising:
a step of providing a speech synthesis dictionary which registers a plurality of element filters obtained by separating the spectrum correction filters formed based on the speech waveform data, and
wherein the correction step includes a step of acquiring the plurality of element filters corresponding to the speech waveform data to be processed in the acquisition step, and applying each of the plurality of obtained element filters to not less than two out of the speech waveform data, the micro-segments, and the superposed waveform data.

### Claim 10 (depends on 1)

The method according to claim 1, wherein the re-arrangement step of re-arranging the micro-segments cut using the window function includes at least one of steps of changing an interval of the micro-segments, repeating a given micro-segment, and skipping the micro-segments.

### Claim 11 (depends on 1)

The method according to claim 1, wherein the re-arrangement step includes a step of skipping the micro-segments, and
the correction step includes a step of applying the correction filter to remaining micro-segments after the skipping process.

### Claim 12 (depends on 1)

The method according to claim 1, wherein the correction step includes a step of using a replacement filter that replaces the spectrum correction filter, and
said method further comprises a providing step of providing, to the acquisition step, modified speech waveform data which is generated by processing the speech waveform data to correct an influence of use of the replacement filter.

### Claim 13 (depends on 12)

The method according to claim 12, wherein the replacement filter is obtained by approximating a correction filter formed based on the speech waveform data.

### Claim 14 (depends on 13)

The method according to claim 13, wherein the providing step includes a step of obtaining the modified speech waveform data by applying a synthetic filter of an inverse filter of the replacement filter and the correction filter to the speech waveform data.

### Claim 15 (depends on 13)

The method according to claim 13, wherein the replacement filter is obtained by generating an FIR filter based on an impulse response of the correction filter, and windowing the FIR filter at a low order.

### Claim 16 (depends on 13)

The method according to claim 13, wherein an order of the replacement filter is lower than the correction filter.

### Claim 17 (depends on 12)

The method according to claim 12, further comprising:
a speech synthesis dictionary which registers the modified speech waveform data generated in the providing step, and formation information of an alternative correction filter to be applied to each speech waveform data, and
wherein the acquisition step includes a step of acquiring micro-segments from speech waveform data read out from the speech synthesis dictionary, and
the correction step includes a step of forming a replacement filter to be used by reading out the formation information of the alternative correction filter corresponding to the speech waveform data from the speech synthesis dictionary.

### Claim 18 (depends on 17)

The method according to claim 17, wherein the speech synthesis dictionary stores clustering information that registers formation information of representative correction filters for respective classes, which are obtained by clustering a plurality of alternative correction filters, and
each of the speech waveform data is modified according to a representative filter to be applied, and is stored together with information indicating a class to which that representative filter belongs.

### Claim 19

A method of generating a dictionary used in a speech synthesis process, comprising:
a generation step of generating a spectrum correction filter on the basis of speech waveform data to be stored in a speech synthesis dictionary; and
a storage step of storing the spectrum correction filter generated in the generation step in correspondence with the speech waveform data.

### Claim 20 (depends on 19)

The method according to claim 19, further comprising a separation step of separating the spectrum correction filter generated in the generation step into a plurality of element filters, and
the storage step includes a step of storing the plurality of element filters in correspondence with the speech waveform data.

### Claim 21

A method of generating a dictionary used in a speech synthesis process, comprising:
a first generation step of generating a spectrum correction filter on the basis of each of speech waveform data to be stored in a speech synthesis dictionary;
a second generation step of generating spectrum-corrected speech waveform data by applying the spectrum correction filter to the corresponding speech waveform data; and
a storage step of storing the spectrum-corrected speech waveform data generated in the second generation step in the speech synthesis dictionary.

### Claim 22

A method of generating a dictionary used in a speech synthesis process, comprising:
a first generation step of generating a replacement filter which replaces a spectrum correction filter obtained based on speech waveform data;
a second generation step of generating modified waveform data by processing the speech waveform data to correct an influence of use of the replacement filter; and
a storage step of storing the replacement filter generated in the first generation step in correspondence with the modified waveform data generated in the second generation step.

### Claim 23 (depends on 22)

The method according to claim 22, wherein the generation step includes a step of generating, as the replacement filter, a filter which approximates the spectrum correction filter obtained based on the speech waveform data.

### Claim 24 (depends on 22)

The method according to claim 22, wherein the first generation step includes a step of clustering a plurality of correction filters into a predetermined number of classes, and generating clustering information that assigns representative correction filters to the respective classes as replacement filters, and
the storage step includes a step of storing the clustering information, and storing class identification information which specifies one of the predetermined number of classes and is assigned to the modified waveform data generated in the second generation step.

### Claim 25

A speech synthesis apparatus comprising:
acquisition means for acquiring micro-segments from speech waveform data and a window function;
re-arrangement means for re-arranging the micro-segments acquired by said acquisition means to change prosody upon synthesis;
synthesis means for outputting synthetic speech waveform data on the basis of superposed waveform data obtained by superposing the micro-segments re-arranged by said re-arrangement means; and
correction means for correcting at least one of the speech waveform data, the micro-segments, and the superposed waveform data using a spectrum correction filter formed based on the speech waveform data to be processed by said acquisition means.

### Claim 26

A generation apparatus of a dictionary used in a speech synthesis process, comprising:
generation means for generating a spectrum correction filter on the basis of speech waveform data to be stored in a speech synthesis dictionary; and
storage means for storing the spectrum correction filter generated by said generation means in correspondence with the speech waveform data.

### Claim 27

A generation apparatus of a dictionary used in a speech synthesis process, comprising:
first generation means for generating a spectrum correction filter on the basis of each of speech waveform data to be stored in a speech synthesis dictionary;
second generation means for generating spectrum-corrected speech waveform data by applying the spectrum correction filter to the corresponding speech waveform data; and
storage means for storing the spectrum-corrected speech waveform data generated by said second generation means in the speech synthesis dictionary.

### Claim 28

A generation apparatus of a dictionary used in a speech synthesis process, comprising:
first generation means for generating a replacement filter which replaces a spectrum correction filter obtained based on speech waveform data;
second generation means for generating modified waveform data by processing the speech waveform data to correct an influence of use of the replacement filter; and
storage means for storing the replacement filter generated by said first generation means in correspondence with the modified waveform data generated by said second generation means.

### Claim 29 (depends on 1)

A computer readable memory storing a control program for making a computer execute a speech synthesis dictionary generation method of claim 1.
