# US 12,763,064 B2 · Timbre selection method and apparatus, electronic device, readable storage medium, and program product

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,763,064 B2
- Document ID: US12763064B2
- Status: Granted
- Kind: B2
- Application number: 18/708,538
- Filed: 2022-11-10
- Published: 2025-01-23
- Granted: 2026-10-13
- Earliest claimed: 2021-11-11
- Assignees: Beijing Zitiao Network Technology Co., Ltd.
- Applicants: Beijing Zitiao Network Technology Co., Ltd.
- Inventors: Keke Wang; Chen Ding; Xudong Mao
- Web page: https://patentgrep.com/patent/US12763064B2

## CPC

- G10L 13/033: Voice editing, e.g. manipulating the voice of the synthesiser
- G10L 13/04: Details of speech synthesis systems, e.g. synthesiser structure or memory management
- G10L 21/0308: characterised by the type of parameter measurement, e.g. correlation techniques, zero crossing techniques or predictive techniques
- G10L 25/18: the extracted parameters being spectral information of each sub-band
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

The present disclosure relates to a timbre selection method and apparatus, an electronic device, a readable storage medium, and a program product. According to the method, a timbre feature of a speech to be matched is obtained by analyzing a spectral feature of the speech to be matched, and then a target sample audio is determined from at least one sample audio based on a similarity between the timbre feature of the speech to be matched and a timbre feature of the at least one sample audio, where a timbre of the target sample audio matches the timbre of the speech to be matched.

## Claims

### Claim 1

A method of implementing automatic timbre selection for audio dubbing, comprising:
performing spectral feature extraction on a speech of a voice role among a plurality of voice roles to obtain a spectral feature of the voice role;
performing timbre feature extraction on the spectral feature of the voice role to obtain a timbre feature of the voice role;
determining a target sample audio from at sample audio library as a dubbing timbre for the voice role based on the timbre feature of the voice role and a timbre feature of at least one initial sample audio in the sample audio library, wherein a timbre of the target sample audio matches the timbre feature of the voice role; and
generating a dubbed audio for the voice role based on the target sample audio that has the same timber style as the speech of the voice role.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the timbre feature comprises a feature in one or more specific dimensions; and the determining a target sample audio from at sample audio library based on the timbre feature of the voice role and a timbre feature of at least one initial sample audio in the sample audio library comprises:
obtaining a similarity between a feature, in the specific dimension, of the timbre of the voice role and the timbre of the at least one initial sample audio in the specific dimension according to a preset order corresponding to the one or more specific dimensions; and
performing step-by-step selection based on the similarity between the feature, in the specific dimension, of the timbre of the voice role and the timbre of the at least one initial sample audio in the specific dimension, to determine the target sample audio from the at least one initial sample audio.

### Claim 3 (depends on 2)

The method according to claim 2, wherein the feature in the one or more specific dimensions comprises at least one of a timbre style feature or a voiceprint feature.

### Claim 4 (depends on 1)

The method according to claim 1, wherein the determining a target sample audio from at sample audio library based on the timbre feature of the voice role and a timbre feature of at least one initial sample audio in the sample audio library comprises:
determining a plurality of candidate sample audios from a plurality of initial sample audios based on the timbre feature of the voice role and timbre features of the plurality of initial sample audios; and
determining the target sample audio from the plurality of candidate sample audios.

### Claim 5 (depends on 1)

The method according to claim 1, wherein before the performing spectral feature extraction on a speech of a voice role, the method further comprises:
performing speech segmentation on an original speech, to obtain at least one speech segment; and
performing clustering on the at least one speech segment, to obtain the one or more speech segment sets, wherein each of the speech segment sets belongs to one voice role, and one of the speech segment sets comprises the speech to be matched.

### Claim 6 (depends on 5)

The method according to claim 5, wherein before the performing speech segmentation on an original speech, the method further comprises:
performing speech separation on an overlapping speech segment in the original speech, to obtain a speech segment corresponding to each voice role in the overlapping speech segment.

### Claim 7 (depends on 1)

The method according to claim 1, wherein the method further comprises:
inputting a text to be dubbed to a speech synthesis model corresponding to the target sample audio, to obtain a target dub output by the speech synthesis model.

### Claim 8

An electronic device, comprising: a memory and a processor;
wherein the memory is configured to store computer program instructions; and the processor is configured to execute the computer program instructions, to implement operations comprising:
performing spectral feature extraction on a speech of a voice role among a plurality of voice roles to obtain a spectral feature of the voice role;
performing timbre feature extraction on the spectral feature of the voice role to obtain a timbre feature of the voice role;
determining a target sample audio from at sample audio library as a dubbing timbre for the voice role based on the timbre feature of the voice role and a timbre feature of at least one initial sample audio in the sample audio library, wherein a timbre of the target sample audio matches the timbre feature of the voice role; and
generating a dubbed audio for the voice role based on the target sample audio that has the same timber style as the speech of the voice role.

### Claim 9 (depends on 8)

The electronic device according to claim 8, wherein the timbre feature comprises a feature in one or more specific dimensions; and the determining a target sample audio from at sample audio library based on the timbre feature of the voice role and a timbre feature of at least one initial sample audio in the sample audio library comprises:
obtaining a similarity between a feature, in the specific dimension, of the timbre of the voice role and the timbre of the at least one initial sample audio in the specific dimension according to a preset order corresponding to the one or more specific dimensions; and
performing step-by-step selection based on the similarity between the feature, in the specific dimension, of the timbre of the voice role and the timbre of the at least one initial sample audio in the specific dimension, to determine the target sample audio from the at least one initial sample audio.

### Claim 10 (depends on 9)

The electronic device according to claim 9, wherein the feature in the one or more specific dimensions comprises at least one of a timbre style feature or a voiceprint feature.

### Claim 11 (depends on 8)

The electronic device according to claim 8, wherein the determining a target sample audio from at sample audio library based on the timbre feature of the voice role and a timbre feature of at least one initial sample audio in the sample audio library comprises:
determining a plurality of candidate sample audios from a plurality of initial sample audios based on the timbre feature of the voice role and timbre features of the plurality of initial sample audios; and
determining the target sample audio from the plurality of candidate sample audios.

### Claim 12 (depends on 8)

The electronic device according to claim 8, wherein before the performing spectral feature extraction on a speech of a voice role, the operations further comprise:
performing speech segmentation on an original speech, to obtain at least one speech segment; and
performing clustering on the at least one speech segment, to obtain the one or more speech segment sets, wherein each of the speech segment sets belongs to one voice role, and one of the speech segment sets comprises the speech to be matched.

### Claim 13 (depends on 8)

The electronic device according to claim 8, wherein the operations further comprise:
inputting a text to be dubbed to a speech synthesis model corresponding to the target sample audio, to obtain a target dub output by the speech synthesis model.

### Claim 14

A non-transitory readable storage medium, comprising: computer program instructions; wherein the computer program instructions, when executed by at least one processor, cause the at least one processor to perform operations comprising:
performing spectral feature extraction on a speech of a voice role among a plurality of voice roles to obtain a spectral feature of the voice role;
performing timbre feature extraction on the spectral feature of the voice role to obtain a timbre feature of the voice role;
determining a target sample audio from at sample audio library as a dubbing timbre for the voice role based on the timbre feature of the voice role and a timbre feature of at least one initial sample audio in the sample audio library, wherein a timbre of the target sample audio matches the timbre feature of the voice role; and
generating a dubbed audio for the voice role based on the target sample audio that has the same timber style as the speech of the voice role.

### Claim 15 (depends on 14)

The non-transitory readable storage medium according to claim 14, wherein the timbre feature comprises a feature in one or more specific dimensions; and the determining a target sample audio from a sample audio library based on the timbre feature of the voice role and a timbre feature of at least one initial sample audio in the sample audio library comprises:
obtaining a similarity between a feature, in the specific dimension, of the timbre of the voice role and the timbre of the at least one initial sample audio in the specific dimension according to a preset order corresponding to the one or more specific dimensions; and
performing step-by-step selection based on the similarity between the feature, in the specific dimension, of the timbre of the voice role and the timbre of the at least one initial sample audio in the specific dimension, to determine the target sample audio from the at least one initial sample audio.

### Claim 16 (depends on 15)

The non-transitory readable storage medium according to claim 15, wherein the feature in the one or more specific dimensions comprises at least one of a timbre style feature or a voiceprint feature.

### Claim 17 (depends on 14)

The non-transitory readable storage medium according to claim 14, wherein the determining a target sample audio from a sample audio library based on the timbre feature of the speech to be matched voice role and a timbre feature of at least one initial sample audio in the sample audio library comprises:
determining a plurality of candidate sample audios from a plurality of initial sample audios based on the timbre feature of the speech to be matched voice role and timbre features of the plurality of initial sample audios; and
determining the target sample audio from the plurality of candidate sample audios.

### Claim 18 (depends on 14)

The non-transitory readable storage medium according to claim 14, wherein before the performing spectral feature extraction on a speech of a voice role, the operations further comprise:
performing speech segmentation on an original speech, to obtain at least one speech segment; and
performing clustering on the at least one speech segment, to obtain the one or more speech segment sets, wherein each of the speech segment sets belongs to one voice role, and one of the speech segment sets comprises the speech to be matched.

### Claim 19 (depends on 18)

The non-transitory readable storage medium according to claim 18, wherein before the performing speech segmentation on an original speech, the operations further comprise:
performing speech separation on an overlapping speech segment in the original speech, to obtain a speech segment corresponding to each voice role in the overlapping speech segment.

### Claim 20 (depends on 14)

The non-transitory readable storage medium according to claim 14, wherein the operations further comprise: inputting a text to be dubbed to a speech synthesis model corresponding to the target sample audio, to obtain a target dub output by the speech synthesis model.