Data-driven audio deepfake detection

A system for generating a visual representation of manipulations in an audio signal, comprising:

• one or more processors;
• one or more memories;
• and one or more programs, wherein the one or more programs are stored in the one or more memories and configured to be executed by the one or more processors, the one or more programs including instructions for:
• inputting the audio signal into a trained machine-learning model, wherein the machine-learning model is trained by:
• generating, based on a training bona fide audio signal, a training bona fide time-frequency representation;
• generating, based on a training spoofed audio signal, a training spoofed time-frequency representation, wherein the training spoofed audio signal is a manipulated version of the training bona fide audio signal;
• generating a training visual representation of manipulations in the training spoofed audio signal based at least on a difference between the training bona fide time-frequency representation and the training spoofed time-frequency representation;
• and
• training the audio deepfake detection machine-learning model based on the training visual representation of the manipulations in the training spoofed audio signal;
• and
• generating, by the machine-learning model, the visual representation of the manipulations in the audio signal.

A non-transitory computer-readable storage medium storing one or more programs for generating a visual representation of manipulations in an audio signal, the one or more programs comprising instructions, which when executed by one or more processors of an electronic device, cause the electronic device to perform:

• inputting the audio signal into a trained machine-learning model, wherein the machine-learning model is trained by:
• generating, based on a training bona fide audio signal, a training bona fide time-frequency representation;
• generating, based on a training spoofed audio signal, a training spoofed time-frequency representation, wherein the training spoofed audio signal is a manipulated version of the training bona fide audio signal;
• generating a training visual representation of manipulations in the training spoofed audio signal based at least on a difference between the training bona fide time-frequency representation and the training spoofed time-frequency representation;
• and
• training the audio deepfake detection machine-learning model based on the training visual representation of the manipulations in the training spoofed audio signal;
• and
• generating, by the machine-learning model, the visual representation of the manipulations in the audio signal.