Techniques for disentangled variational speech representation learning for zero-shot voice conversion

An apparatus for disentangled variational speech representation learning for voice conversion, the apparatus comprising:

• at least one memory configured to store program code;
• and
• at least one processor configured to read the program code and operate as instructed by the program code, the program code including:
• receiving code configured to cause the at least one processor to receive input speech segments;
• first encoding code configured to cause the at least one processor to encode the input speech segments via a shared encoder to generate a speaker embedding and a content embedding;
• second encoding code configured to cause the at least one processor to encode posterior distributions of the speaker embedding via a speaker encoder and encode posterior distributions of the content embedding via a content encoder to obtain encoded results;
• and
• decoding code configured to cause the at least one processor to decode the encoded results by concatenating the encoded results to obtain a reconstructed speech output,
• wherein the shared encoder includes:
• a 2-layer meridian lossless packing (MLP) with a first hidden size, and
• convolutional layers with a plurality of channels of which each of the convolutional layers is followed by a linear layer and an Instancenorm2D layer.

A non-transitory computer readable medium storing instructions, the instructions comprising:

• one or more instructions that, when executed by at least one processor of an apparatus for disentangled variational speech representation learning for voice conversion storing instructions that, cause the at least one processor to:
• receive input speech segments;
• encode the input speech segments via a shared encoder to generate a speaker embedding and a content embedding;
• encode posterior distributions of the speaker embedding via a speaker encoder and encode posterior distributions of the content embedding via a content encoder to obtain encoded results;
• and
• decode the encoded results by concatenating the encoded results to obtain a reconstructed speech output,
• wherein the shared encoder includes:
• a 2-layer meridian lossless packing (MLP) with a first hidden size, and
• convolutional layers with a plurality of channels of which each of the convolutional layers is followed by a linear layer and an Instancenorm2D layer.