Speech conversion method and apparatus, storage medium, and electronic device

A speech conversion apparatus comprising:

• a first acquisition module configured to acquire a source speech to be converted and a target speech sample of a target speaker;
• a first extraction module configured to recognize a style category of the target speech sample by an audio feature encoding module, and extract a target audio feature from the target speech sample according to the style category of the target speech sample, wherein the target audio feature includes a textual feature, a prosodic feature and a timbre feature of the target speech sample;
• a second extraction module configured to extract a source audio feature from the source speech by the audio feature encoding module, wherein the source audio feature includes a textual feature, a prosodic feature and a timbre feature of the source speech;
• a second acquisition module configured to acquire a first style feature of the target speech sample by a style feature encoding module and determine a second style feature of the target speech sample according to the first style feature, wherein the first style feature is used to indicate a static voice characteristic of the target speech sample, and the second style feature is used to indicate predicted values for a feature bias amount and a gain amount of the first style feature within a preset duration;
• a fusing and mapping module configured to fuse and map the source audio feature of the source speech, the target audio feature of the target speech sample, and the second style feature of the target speech sample to obtain a joint encoding feature;
• a conversion module configured to decode the joint encoding feature on which a standard streaming operation is performed, to obtain a target speech feature corresponding to a speaking style of the target speaker;
• and convert the source speech based on the target speech feature to obtain a target speech;
• a training module configured to:
• train a first clustering model by using first training samples, wherein the first training samples include speech samples of a plurality of speakers, and the speech samples of the plurality of speakers correspond to different style types, and wherein the first clustering model is configured for clustering the first training samples and determining, according to a result of the clustering, category labels corresponding to the first training samples;
• train a second clustering model by using second training samples, wherein the second training samples include speech samples of a plurality of speakers, and the speech samples of the plurality of speakers correspond to different style types, and wherein the second clustering model is configured for clustering the second training samples and determining, according to a result of the clustering, category labels corresponding to the second training samples, the first clustering model and the second clustering model using different structures of feature extractors to perform clustering from different dimensions;
• input third training samples into the trained first clustering model, the trained second clustering model, and an initial audio feature encoding module, wherein the third training samples include speech samples of a plurality of speakers;
• and
• train the initial audio feature encoding module according to a loss function for the initial audio feature encoding module, and training the initial audio feature encoding module to convergence according to actual category labels output from the first clustering model and the second clustering model and the predicted category labels output from the initial audio feature encoding module to obtain the audio feature encoding module, wherein the audio feature encoding module is configured for performing audio feature extraction based on the style type of speech.