Audio spoof detection using attention-based contrastive learning

A system for detecting fake audios, comprising:

• one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for:
• converting audio data into an image representation of the audio data;
• providing the image representation of the audio data to a trained machine-learning model, wherein the trained machine-learning model comprises a trained patch split component for splitting the image representation of the audio data into a sequence of image patches, the machine learning model:
• generating, using a trained self-attention branch, one or more representation embeddings based on the image representation of the audio data;
• and
• receiving, using a trained classifier component, the one or more representation embeddings and outputting a classification result;
• and
• providing the classification result.

A non-transitory computer-readable storage medium storing one or more programs for detecting fake audios, the one or more programs comprising instructions, which when executed by one or more processors of an electronic device having a display, cause the electronic device to:

• convert audio data into an image representation of the audio data;
• provide the image representation of the audio data to a trained machine-learning model, wherein the trained machine-learning model comprises a trained patch split component for splitting the image representation of the audio data into a sequence of image patches, the machine learning model:
• generating, using a trained self-attention branch, one or more representation embeddings based on the image representation of the audio data;
• and
• receiving, using a trained classifier component, the one or more representation embeddings and outputting a classification result;
• and
• provide the classification result.