Text-based virtual object animation generation method, apparatus, storage medium, and terminal

A non-transitory storage medium, which is configured to store computer programs, wherein when executed by a processor, the computer programs cause the processor to perform the following steps:

• acquiring text information, wherein the text information comprises an original text of a virtual object animation to be generated;
• analyzing an emotional feature and a rhyme boundary of the text information;
• performing speech synthesis according to the emotional feature, the rhyme boundary, and the text information to obtain audio information, wherein the audio information comprises emotional speech obtained by conversion based on the original text;
• and
• generating a corresponding virtual object animation based on the text information and the audio information, wherein the virtual object animation is synchronized in time with the audio information;
• wherein generating the corresponding virtual object animation based on the text information and the audio information comprises:
• receiving input information, wherein the input information comprises the text information and the audio information;
• converting the audio information into pronunciation units and time codes based on speech recognition technology and a preset pronunciation dictionary, wherein the text information is used for determining a duration of each piece of speech in the audio information;
• and
• performing a time alignment operation on the pronunciation units according to the time codes so as to obtain a time-aligned pronunciation unit sequence;
• performing a feature analysis on the time-aligned pronunciation unit sequence to obtain a corresponding linguistic feature sequence;
• and
• inputting the linguistic feature sequence into a preset temporal sequence mapping model to generate the corresponding virtual object animation based on the linguistic feature sequence;
• wherein performing the speech synthesis based on the emotional feature, the rhyme boundary, and the text information to obtain the audio information comprises:
• inputting the text information, the emotional feature, and the rhyme boundary into a preset speech synthesis model, wherein the preset speech synthesis model is used for converting an inputted text sequence into a speech sequence in a temporal sequence, and speech in the speech sequence carries emotion of the text at a corresponding point in time;
• and
• acquiring the audio information outputted by the preset speech synthesis model.

A terminal, comprising a memory and a processor, wherein the memory stores a computer program capable of running on the processor, and the processor, when executing the computer program, performs the following steps:

• acquiring text information, wherein the text information comprises an original text of a virtual object animation to be generated;
• analyzing an emotional feature and a rhyme boundary of the text information;
• performing speech synthesis according to the emotional feature, the rhyme boundary, and the text information to obtain audio information, wherein the audio information comprises emotional speech obtained by conversion based on the original text;
• and
• generating a corresponding virtual object animation based on the text information and the audio information, wherein the virtual object animation is synchronized in time with the audio information;
• wherein generating the corresponding virtual object animation based on the text information and the audio information comprises:
• receiving input information, wherein the input information comprises the text information and the audio information;
• converting the audio information into pronunciation units and time codes based on speech recognition technology and a preset pronunciation dictionary, wherein the text information is used for determining a duration of each piece of speech in the audio information;
• and
• performing a time alignment operation on the pronunciation units according to the time codes so as to obtain a time-aligned pronunciation unit sequence;
• performing a feature analysis on the time-aligned pronunciation unit sequence to obtain a corresponding linguistic feature sequence;
• and
• inputting the linguistic feature sequence into a preset temporal sequence mapping model to generate the corresponding virtual object animation based on the linguistic feature sequence;
• wherein performing the speech synthesis based on the emotional feature, the rhyme boundary, and the text information to obtain the audio information comprises:
• inputting the text information, the emotional feature, and the rhyme boundary into a preset speech synthesis model, wherein the preset speech synthesis model is used for converting an inputted text sequence into a speech sequence in a temporal sequence, and speech in the speech sequence carries emotion of the text at a corresponding point in time;
• and
• acquiring the audio information outputted by the preset speech synthesis model.