US 11,538,456 B2Granted

Audio file processing method, electronic device, and storage medium

AssigneeTencent Technology (Shenzhen) Company Limited

InventorChunjiang Lai

Life of this patent

Estimating the term…

Loading patent…

Abstract

An audio file processing method is provided for an electronic device. The method includes extracting at least one audio segment from a first audio file, recognizing at least one to-be-replaced audio segment representing a target role from the at least one audio segment, and determining time frame information of each to-be-replaced audio segment in the first audio file. The method also includes obtaining to-be-dubbed audio data for each to-be-replaced audio segment, and replacing data in the to-be-replaced audio segment with the to-be-dubbed audio data according to the time frame information, to obtain a second audio file. The at least one to-be-replaced audio segment is divided from the at least one audio segment based on a structure and a word count in a sentence corresponding to each to-be-replaced audio segment.

Claims4

Dependent claims

An audio file processing method via a user interface of a terminal in communication with a processing device, the method comprising:

receiving, via the user interface of the terminal, a voice replacement request for a target role in a first audio file, by:
receiving, via the user interface of the terminal, a right-click trigger on the user interface;
displaying, on the user interface, a pop up window, the pop up window showing a list of actor roles whose voice is replaceable;
and
receiving, via the pop up window on the user interface, a user selection of the target role from the list of actor roles as displayed;
sending, by the terminal, the voice replacement request to the processing device, the voice replacement request including an identifier of the first audio file and an identifier of the target role as selected;
receiving, by the processing device, the voice replacement request from the terminal;
determining, by the processing device, time frame information of a to-be-replaced audio segment in the first audio file according to the voice replacement request, by:
extracting, by the processing device, first candidate time frame information from the to-be-replaced audio segment based on a short sentence division principle, and obtaining a first start time t1 in the first candidate time frame information;
extracting, by the processing device, second candidate time frame information from the to-be-replaced audio segment based on a long sentence division principle, and obtaining a first start time t2 in the second candidate time frame information;
and
setting an average of t1 and t2 as the time frame information of the to-be-replaced audio segment;
replacing data in the to-be-replaced audio segment with to-be-dubbed audio data according to the time frame information, to obtain a second audio file;
and
sending, by the processing device, the second audio file to the terminal.

12 more elements

A terminal, comprising:

a processor;
and a memory, the memory storing computer instructions executable by the processor, wherein the processor is configured to execute the computer instruction to perform a method via a user interface of the terminal in communication with a processing device, the method including:
receiving, via the user interface of the terminal, a voice replacement request for a target role in a first audio file, by:
receiving, via the user interface of the electronic device terminal, a right-click trigger on the user interface;
displaying, on the user interface, a pop up window, the pop up window showing a list of actor roles whose voice is replaceable;
and
receiving, via the pop up window on the user interface, a user selection of the target role from the list of actor roles as displayed;
sending, by the terminal, the voice replacement request to the processing device, the voice replacement request including an identifier of the first audio file and an identifier of the target role as selected;
receiving, by the processing device, the voice replacement request from the terminal;
determining, by the processing device, time frame information of a to-be-replaced audio segment in the first audio file according to the voice replacement request, by:
extracting, by the processing device, first candidate time frame information from the to-be-replaced audio segment based on a short sentence division principle, and obtaining a first start time t11 in the first candidate time frame information;
extracting, by the processing device, second candidate time frame information from the to-be-replaced audio segment based on a long sentence division principle, and obtaining a first start time t21 in the second candidate time frame information;
and
setting an average of t11 and t21 as the time frame information of the to-be-replaced audio segment;
replacing data in the to-be-replaced audio segment with to-be-dubbed audio data according to the time frame information, to obtain a second audio file;
and
sending, by the processing device, the second audio file to the terminal.

14 more elements

A non-transitory computer-readable storage medium storing computer program instructions executable by at least one processor of a terminal to perform a method via a user interface of the terminal in communication with a processing device, the method including:

receiving, via the user interface of the terminal, a voice replacement request for a target role in a first audio file, by:
receiving, via the user interface of the terminal, a right-click trigger on the user interface;
displaying, on the user interface, a pop up window, the pop up window showing a list of actor roles whose voice is replaceable;
and
receiving, via the pop up window on the user interface, a user selection of the target role from the list of actor roles as displayed;
sending, by the terminal, the voice replacement request to the processing device, the voice replacement request including an identifier of the first audio file and an identifier of the target role as selected;
receiving, by the processing device, the voice replacement request from the terminal;
determining, by the processing device, time frame information of a to-be-replaced audio segment in the first audio file according to the voice replacement request, by:
extracting, by the processing device, first candidate time frame information from the to-be-replaced audio segment based on a short sentence division principle, and obtaining a first start time t11 in the first candidate time frame information;
extracting, by the processing device, second candidate time frame information from the to-be-replaced audio segment based on a long sentence division principle, and obtaining a first start time t21 in the second candidate time frame information;
and
setting an average of t11 and t21 as the time frame information of the to-be-replaced audio segment;
replacing data in the to-be-replaced audio segment with to-be-dubbed audio data according to the time frame information, to obtain a second audio file;
and
sending, by the processing device, the second audio file to the terminal.

12 more elements

Dependencies are inferred from the claim text.

Facts

Application: 16/844,283Filed 9 Apr 2020 · continuation
Priority: CN 201711076391.56 Nov 2017
Published: US 2020/0234689 A123 Jul 2020
Granted: 27 Dec 2022
Term ends: Not estimatedThe source records don’t support an estimate.

Classification13 codes

G10L 13/033
Methods for producing synthetic speech; Speech synthesisers: voice editing, e.g. manipulating the voice of the synthesiser
G10L 13/04
Methods for producing synthetic speech; Speech synthesisers: details of speech synthesis systems, e.g. synthesiser structure or memory management
G10L 21/003
Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its…:…
G11B 27/036
Electronic editing of digitised analogue information signals, e.g. audio or video signals: insert-editing