# US 12,763,080 B2 · System and method for detecting deep fake audio > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,763,080 B2 - Document ID: US12763080B2 - Status: Granted - Kind: B2 - Application number: 18/786,830 - Filed: 2024-07-29 - Published: 2026-01-29 - Granted: 2026-10-13 - Earliest claimed: 2024-07-29 - Assignees: Bank of America Corporation - Applicants: Bank of America Corporation - Inventors: Jack Bishop; Jason C. Starin; Carrie E. Gates - Web page: https://patentgrep.com/patent/US12763080B2 ## CPC - G10L 17/12: Score normalisation - G10L 21/0216: characterised by the method used for estimating noise - G10L 2021/02168: {the estimation exclusively taking place during speech pauses} - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING ## Abstract A system for analyzing audio includes a memory configured to store known digital audio representation containing known fraudulent audio streams and a processor operably coupled to the memory. The processor receives a portion of an audio stream from an external device and produces a transcript of the portion of the audio stream. The processor then determines a timing score, an emotional score, a background score, and a content score by analyzing the portion of an audio stream and the corresponding transcript and comparing them to the known digital audio representations and transcripts. The processor then determines if the audio stream is malicious by combining the timing score, emotional score, background score, and content score to produce a combined score and comparing the combined score to a threshold. The processor notifies a user that the call may be fraudulent when the combined score is greater than the threshold. ## Claims ### Claim 1 A system for analyzing audio, comprising: a memory configured to store known digital audio representations, wherein the known digital audio representations comprise two or more portions of fraudulent audio streams; and a processor operably coupled to the memory and configured to: receive a portion of an audio stream from an external device; generate a transcript of the portion of the audio stream by utilizing a long short-term memory (LSTM) neural network; determine a timing score by analyzing a timing of the portion of the audio stream and comparing it to labeled timing of the known digital audio representations, wherein analyzing the timing comprises determining a length of pauses between syllables in the portion of the audio stream; determine an emotional score by analyzing an emotional content of the portion of the audio stream and comparing it to labeled emotional content of the known digital audio representations, wherein the emotional content is determined at least by analyzing the portion of the audio stream to determine which words are emphasized in the portion of the audio stream; identify speech from the transcript of the portion of the audio stream, wherein the audio stream comprises a first portion and a second portion and wherein the first portion comprises the speech and the second portion comprises background noise that is repetitive in nature; remove the first portion from the portion of the audio stream; in response to removing the first portion from the portion of the audio stream, determine a background score by analyzing the second portion of the audio stream to detect the background noise and comparing the detected background noise to known background noise contained in the known digital audio representations; determine a content score using the transcript by comparing the transcript to transcripts produced for the known digital audio representations; determine if the audio stream is malicious by combining the timing score, emotional score, background score, and content score to produce a combined score and comparing the combined score to a threshold; and notify a user when the combined score is greater than the threshold. ### Claim 2 (depends on 1) The system of claim 1, wherein the audio stream is received from the external device and comprises real-time audio. ### Claim 3 (depends on 2) The system of claim 2, wherein the external device is a mobile phone, and the audio stream is an unexpected call received by the user. ### Claim 4 (depends on 1) The system of claim 1, wherein the combined score is a weighted score comprising predetermined weights for each of the timing score, emotional score, background score, and content score and wherein the predetermined weights are determined by analyzing the known digital audio representations using machine learning. ### Claim 5 (depends on 4) The system of claim 4, wherein the machine learning utilizes logistic regression to determine a weight to apply to each of the timing score, emotional score, background score, and content score. ### Claim 6 (depends on 1) The system of claim 1, wherein the timing score, emotional score, background score, and content score are indications of a probability that the portion of the audio stream was produced electronically. ### Claim 7 (depends on 1) The system of claim 1, wherein the timing score is further determined by identifying a speaker in the portion of the audio stream and comparing the portion of the audio stream to known recordings of the speaker that is similar to the portion of the audio stream. ### Claim 8 (depends on 1) The system of claim 1, wherein the speech is removed using the transcript to identify the speech. ### Claim 9 (depends on 1) The system of claim 1, wherein the timing score, emotional score, background score, and content score are determined using machine learning to analyze the portion of the audio stream and the transcript. ### Claim 10 (depends on 1) The system of claim 1, wherein the background noise includes saliva noises and the background score is determined at least in part based on a frequency of the saliva noises. ### Claim 11 A method for communicating: receiving a portion of an audio stream from an external device; generating a transcript of the portion of the audio stream by utilizing a long short-term memory (LSTM) neural network; determining a timing score by analyzing a timing of the portion of the audio stream and comparing it to labeled timing of a known digital audio representations, wherein analyzing the timing comprises determining a length of pauses between syllables in the portion of the audio stream; determining an emotional score by analyzing an emotional content of the portion of the audio stream and comparing it to labeled emotional content of the known digital audio representations, wherein the emotional content is determined at least by analyzing the portion of the audio stream to determine which words are emphasized in the portion of the audio stream; identifying speech from the transcript of the portion of the audio stream, wherein the audio stream comprises a first portion and a second portion and wherein the first portion comprises the speech and the second portion comprises background noise that is repetitive in nature; removing the first portion from the portion of the audio stream; in response to removing the first portion from the portion of the audio stream, determining a background score by analyzing the second portion of the audio stream to detect the background noise and comparing the detected background noise to known background noise contained in the known digital audio representations; determining a content score using the transcript by comparing the transcript to transcripts produced for the known digital audio representations; determining if the audio stream is malicious by combining the timing score, emotional score, background score, and content score to produce a combined score and comparing the combined score to a threshold; and notifying a user when the combined score is greater than the threshold. ### Claim 12 (depends on 11) The method of claim 11, wherein the combined score is a weighted score comprising predetermined weights for each of the timing score, emotional score, background score, and content score and wherein the predetermined weights are determined by analyzing the known digital audio representations using machine learning. ### Claim 13 (depends on 12) The method of claim 12, wherein the machine learning utilizes logistic regression to determine a weight to apply to each of the timing score, emotional score, background score, and content score. ### Claim 14 (depends on 11) The method of claim 11, wherein the timing score, emotional score, background score, and content score are indications of a probability that the portion of the audio stream was produced electronically. ### Claim 15 (depends on 11) The method of claim 11, wherein the speech is removed using the transcript to identify the speech. ### Claim 16 (depends on 11) The method of claim 11, wherein the timing score, emotional score, background score, and content score are determined using machine learning to analyze the portion of the audio stream and the transcript. ### Claim 17 A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to: receive a portion of an audio stream from an external device; generate a transcript of the portion of the audio stream by utilizing a long short-term memory (LSTM) neural network; determine a timing score by analyzing a timing of the portion of the audio stream and comparing it to labeled timing of a known digital audio representations, wherein analyzing the timing comprises determining a length of pauses between syllables in the portion of the audio stream; determine an emotional score by analyzing an emotional content of the portion of the audio stream and comparing it to labeled emotional content of the known digital audio representations, wherein the emotional content is determined at least by analyzing the portion of the audio stream to determine which words are emphasized in the portion of the audio stream; identify speech from the transcript of the portion of the audio stream, wherein the audio stream comprises a first portion and a second portion and wherein the first portion comprises the speech and the second portion comprises background noise that is repetitive in nature; remove the first portion from the portion of the audio stream; in response to removing the first portion from the portion of the audio stream, determine a background score by analyzing the second portion of the audio stream to detect the background noise and comparing the detected background noise to known background noise contained in the known digital audio representations; determine a content score using the transcript by comparing the transcript to transcripts produced for the known digital audio representations; determine if the audio stream is malicious by combining the timing score, emotional score, background score, and content score to produce a combined score and comparing the combined score to a threshold; and notify a user when the combined score is greater than the threshold. ### Claim 18 (depends on 17) The non-transitory computer-readable medium of claim 17, wherein the combined score is a weighted score comprising predetermined weights for each of the timing score, emotional score, background score, and content score and wherein the predetermined weights are determined by analyzing the known digital audio representations using machine learning. ### Claim 19 (depends on 17) The non-transitory computer-readable medium of claim 17, wherein the timing score, emotional score, background score, and content score are indications of a probability that the portion of the audio stream was produced electronically. ### Claim 20 (depends on 17) The non-transitory computer-readable medium of claim 17, wherein the timing score is further determined by identifying a speaker in the portion of the audio stream and comparing the portion of the audio stream to known recordings of the speaker that is similar to the portion of the audio stream.