Video transformer for deepfake detection with incremental learning

A DeepFake detection system, comprising:

• an artificial intelligence (AI) workstation;
• a plurality of client computers connected to the AI workstation,
• wherein the AI workstation includes program instructions which when executed by processing circuitry having multiple processing cores, each processing core performs a method comprising:
• inputting a sequence of video frames of a video;
• detecting faces frame by frame in the video to obtain consecutive face images;
• creating UV texture maps from the face images;
• inputting both the face images and corresponding said UV texture maps;
• extracting image feature maps, by a convolution neural network (CNN) backbone, from the face images and corresponding said UV texture maps and forming an input data structure;
• receiving the input data structure, by a video transformer model that includes multiple encoders;
• computing, by the video transformer model, a classification of the video as being Real or Fake;
• and
• the plurality of client computers each interacting with the AI workstation in parallel to detect whether a video being played in the respective client computer is Real or Fake and display an indication of the detection.

An embedded DeepFake detector that stores program instructions which when executed by processing circuitry having multiple processing cores, each processing core, in parallel with others of the multiple processing cores, performs a method comprising:

• receiving a sequence of video frames of a video captured from a streaming service;
• detecting faces frame by frame in the video to obtain consecutive face images;
• creating UV texture maps from the face images;
• inputting both the face images and corresponding created UV texture maps;
• extracting image feature maps, by a convolution neural network (CNN) backbone, from the face images and corresponding created UV texture maps and forming an input data structure;
• receiving the input data structure, by a video transformer model consisting of multiple encoders;
• computing, by the video transformer model, a classification of the video as being Real or Fake;
• and
• performing an action on the video depending on whether the video is detected as being Real or Fake.