Deepfake detection in data at rest and real-time network traffic

A deepfake detection system for detecting deepfake in live network traffic communications and stored media, the deepfake detection system comprising:

• one or more processors, and
• memory coupled with the one or more processors, the memory configured to store instructions that when executed by the one or more processors cause the one or more processors to:
• extract, by a mid-link server, a visual data and an audio of a plurality of users of an enterprise;
• generate, by the mid-link server, digital fingerprints from the extracted visual data and the audio;
• match, by the mid-link server, the generated digital fingerprints with pre-stored normalized fingerprints to produce an output of the match;
• analyze, by the mid-link server, the extracted visual data and the audio using an artificial intelligence (AI)-detection model;
• generate, by the mid-link server, a deepfake detection score based on the analysis by the AI-detection model;
• generate, by the mid-link server, a fingerprint mismatch flag based on the output of the matching;
• generate, by the mid-link server, a deepfake machine learning flag based on the deepfake detection score;
• combine, by the mid-link server, the output of the match and the deepfake detection score to determine a level of deepfake in the extracted visual data and the audio;
• apply, by the mid-link server, a plurality of policies in real-time based on the level of deepfake in the extracted visual data and the audio, wherein applying the plurality of policies includes blocking access of enterprise content to a user based on a user role in the enterprise, upon triggering of the fingerprint mismatch flag and the deepfake machine learning flag;
• and
• where in the plurality of policies are predefined in a policy store of the enterprise based on the user role and when the user role corresponds to a Trainee in the enterprise and the level of deepfake is indicated by:
• (i) the fingerprint mismatch flag, a policy comprises holding access of the enterprise content to the user and re-authenticating the user,
• (ii) the deepfake machine learning flag, the policy comprises holding access of the enterprise content until an administrator of the enterprise contacts the user, or
• (iii) both the fingerprint mismatch flag and the deepfake machine learning flag, the policy comprises blocking user access of the enterprise content and sending an alert to a user device of the user and an administration of the enterprise.

A non-transitory computer-readable storage medium having stored thereon instructions for causing at least one computer to facilitate a method for detecting deepfake in live network traffic communications and stored media, the method comprising:

• extracting visual data and audio of a plurality of users of an enterprise;
• generating digital fingerprints from the extracted visual data and the audio;
• matching the generated digital fingerprints with pre-stored normalized fingerprints to produce an output of the matching;
• analyzing the extracted visual data and the audio using an artificial intelligence (AI)-detection model;
• generating a deepfake detection score based on the analysis by the AI-detection model;
• generating a fingerprint mismatch flag based on the output of the matching;
• generating a deepfake machine learning flag based on the deepfake detection score;
• combining the output of the matching and the deepfake detection score to determine a level of deepfake in the extracted visual data and the audio;
• applying a plurality of policies in real-time based on the level of deepfake in the extracted visual data and the audio, wherein applying the plurality of policies includes blocking access of enterprise content to a user based on a user role in the enterprise, upon triggering of the fingerprint mismatch flag and the deepfake machine learning flag;
• and
• wherein the plurality of policies are predefined in a policy store of the enterprise based on the user role and when the user role corresponds to a Trainee in the enterprise and the level of deepfake is indicated by:
• (i) the fingerprint mismatch flag, a policy comprises holding access of the enterprise content to the user and re-authenticating the user,
• (ii) the deepfake machine learning flag, the policy comprises holding access of the enterprise content until an administrator of the enterprise contacts the user, or
• (iii) both the fingerprint mismatch flag and the deepfake machine learning flag, the policy comprises blocking user access of the enterprise content and sending an alert to a user device of the user and an administration of the enterprise.