Synthetic speech detection

A computerized method for synthetic speech detection (SSD), the computerized method comprising:

• performing a training operation, the training operation comprising:
• receiving an input sample comprising audio;
• extracting a plurality of acoustic features corresponding to speech in the audio;
• processing the plurality of extracted acoustic features in parallel using a plurality of neural networks to output abstracted feature vectors, wherein the plurality of neural networks includes at least deep neural networks (DNNs) having a shared output used to perform a multitask training of an SSD model and at least one auxiliary classification model selected from a speaker classification model and a channel classification model;
• generating, via the plurality of neural networks, a pooled feature vector from the output abstracted feature vectors using a pooling operation, wherein the pooling operation comprises multi-head attentive pooling that assigns to each of the output abstracted feature vectors, a respective attention weight, used to generate the pooled feature vector, and wherein the pooled feature vector is a single vector;
• performing the multitask training at substantially a same time, using the pooled feature vector, the multitask training including an SSD task of the SSD model and at least one auxiliary classification task selected from a speaker classification task of the speaker classification model and a channel classification task of the channel classification model;
• and
• during an inference operation subsequent to the training operation, detecting synthetic speech using at least the SSD task of the SSD model.

A system for synthetic speech detection (SSD), the system comprising:

• at least one processor;
• and
• at least one memory comprising computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the at least one processor to:
• perform a training operation, the training operation comprising:
• receive an input sample comprising audio;
• extract a plurality of acoustic features corresponding to speech in the audio;
• process the plurality of extracted acoustic features in parallel using a plurality of neural networks to output abstracted feature vectors, wherein the plurality of neural networks includes at least deep neural networks (DNNs) having a shared output used to perform a multitask training of an SSD model and at least one auxiliary classification model selected from a speaker classification model and a channel classification model;
• generate, via the plurality of neural networks, a pooled feature vector from the output abstracted feature vectors using a pooling operation, wherein the pooling operation comprises multi-head attentive pooling that assigns to each of the output abstracted feature vectors, a respective attention weight, used to generate the pooled feature vector, and wherein the pooled feature vector is a single vector;
• perform the multitask training at substantially a same time, using the pooled feature vector, the multitask training including an SSD task of the SSD model and at least one auxiliary classification task selected from a speaker classification task of the speaker classification model and a channel classification task of the channel classification model;
• and
• during an inference operation subsequent to the training operation, detect synthetic speech using at least the SSD task of the SSD model.

One or more computer storage media having computer-executable instructions for synthetic speech detection (SSD) that upon execution by a processor, cause the processor to:

• perform a training operation, the training operation comprising:
• receive an input sample comprising audio;
• extract a plurality of acoustic features corresponding to speech in the audio;
• process the plurality of extracted acoustic features in parallel using a plurality of neural networks to output abstracted feature vectors, wherein the plurality of neural networks includes at least deep neural networks (DNNs) having a shared output used to perform a multitask training of an SSD model and at least one auxiliary classification model selected from a speaker classification model and a channel classification model;
• generate, via the plurality of neural networks, a pooled feature vector from the output abstracted feature vectors using a pooling operation, wherein the pooling operation comprises multi-head attentive pooling that assigns to each of the output abstracted feature vectors, a respective attention weight, used to generate the pooled feature vector, and wherein the pooled feature vector is a single vector;
• perform the multitask training at substantially a same time, using the pooled feature vector, the multitask training including an SSD task of the SSD model and at least one auxiliary classification task selected from a speaker classification task of the speaker classification model and a channel classification task of the channel classification model;
• and
• during an inference operation subsequent to the training operation, detect synthetic speech using at least the SSD task of the SSD model.