System and method for detecting deep fake audio

A method for communicating:

• receiving a portion of an audio stream from an external device;
• producing a transcript of the portion of the audio stream;
• determining a timing score by analyzing a timing of the portion of the audio stream and comparing it to labeled timing of a known digital audio representations, wherein analyzing the timing comprises determining a length of pauses between syllables in the portion of the audio stream;
• determining an emotional score by analyzing an emotional content of the portion of the audio stream and comparing it to labeled emotional content of the known digital audio representations, wherein the emotional content is determined at least by analyzing the portion of the audio stream to determine which words are emphasized in the portion of the audio stream;
• determining a background score by analyzing the audio stream to detect background noise and comparing the detected background noise to known background noise contained in the known digital audio representations;
• determining a content score using the transcript by comparing the transcript to transcripts produced for the known digital audio representations;
• determining if the audio stream is malicious by combining the timing score, emotional score, background score, and content score to produce a combined score and comparing the combined score to a threshold;
• and
• notifying a user when the combined score is greater than the threshold.

A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to:

• receive a portion of an audio stream from an external device;
• produce a transcript of the portion of the audio stream;
• determine a timing score by analyzing a timing of the portion of the audio stream and comparing it to labeled timing of a known digital audio representations, wherein analyzing the timing comprises determining a length of pauses between syllables in the portion of the audio stream;
• determine an emotional score by analyzing an emotional content of the portion of the audio stream and comparing it to labeled emotional content of the known digital audio representations, wherein the emotional content is determined at least by analyzing the portion of the audio stream to determine which words are emphasized in the portion of the audio stream;
• determine a background score by analyzing the audio stream to detect background noise and comparing the detected background noise to known background noise contained in the known digital audio representations;
• determine a content score using the transcript by comparing the transcript to transcripts produced for the known digital audio representations;
• determine if the audio stream is malicious by combining the timing score, emotional score, background score, and content score to produce a combined score and comparing the combined score to a threshold;
• and
• notify a user when the combined score is greater than the threshold.