Speech translation method, device, and storage medium

An electronic device, comprising a processor and a memory storing computer programs, wherein the computer programs, when executed by the processor, implement the following:

• extracting, through an encoder of an end-to-end speech translation model, a semantic feature of a to-be-processed speech;
• decoding, through a decoder of the end-to-end speech translation model, a source language text corresponding to the semantic feature from the semantic feature;
• decoding, through the decoder of the end-to-end speech translation model, the semantic feature according to the source language text to obtain a text sequence corresponding to the semantic feature, wherein the text sequence comprises the source language text and a target language text corresponding to the source language text;
• and
• splitting the text sequence to obtain the target language text corresponding to the to-be-processed speech,
• wherein a training process of the end-to-end speech translation model comprises:
• pre-training, according to a text translation sample, the decoder of the end-to-end speech translation model to obtain an initial decoder;
• and
• initializing the decoder of the end-to-end speech translation model based on the initial decoder;
• and training the initialized end-to-end speech translation model,
• wherein the text translation sample comprises a source language sample text and a target language sample text corresponding to the source language sample text, and
• wherein pre-training, according to the text translation sample, the decoder of the end-to-end speech translation model to obtain the initial decoder comprises:
• splicing the source language sample text and the target language sample text to obtain a spliced sample sequence;
• and
• pre-training, based on a masked cross-entropy loss function, the decoder by using the source language sample text and an all-zero vector as input of the decoder of the end-to-end speech translation model and the spliced sample sequence as desired output to obtain the initial decoder, wherein the masked cross-entropy loss function is configured to mask a prediction loss of a source language prediction text corresponding to the all-zero vector.

A non-transitory computer-readable storage medium storing computer programs, wherein the computer programs, when executed by a processor, implement the following:

• extracting, through an encoder of an end-to-end speech translation model, a semantic feature of a to-be-processed speech;
• decoding, through a decoder of the end-to-end speech translation model, a source language text corresponding to the semantic feature from the semantic feature;
• decoding, through the decoder of the end-to-end speech translation model, the semantic feature according to the source language text to obtain a text sequence corresponding to the semantic feature, wherein the text sequence comprises the source language text and a target language text corresponding to the source language text;
• and
• splitting the text sequence to obtain the target language text corresponding to the to-be-processed speech,
• wherein a training process of the end-to-end speech translation model comprises:
• pre-training, according to a text translation sample, the decoder of the end-to-end speech translation model to obtain an initial decoder;
• and
• initializing the decoder of the end-to-end speech translation model based on the initial decoder;
• and training the initialized end-to-end speech translation model,
• wherein the text translation sample comprises a source language sample text and a target language sample text corresponding to the source language sample text, and
• wherein pre-training, according to the text translation sample, the decoder of the end-to-end speech translation model to obtain the initial decoder comprises:
• splicing the source language sample text and the target language sample text to obtain a spliced sample sequence;
• and
• pre-training, based on a masked cross-entropy loss function, the decoder by using the source language sample text and an all-zero vector as input of the decoder of the end-to-end speech translation model and the spliced sample sequence as desired output to obtain the initial decoder, wherein the masked cross-entropy loss function is configured to mask a prediction loss of a source language prediction text corresponding to the all-zero vector.