Fully convolutional transformer based generative adversarial networks

A method for detecting anomaly in video data, comprising:

• receiving, at a generator, past video frames, Wherein the generator includes fully convolutional transformer based generative adversarial networks (FCT-GANs)
• extracting, by the generator, spatio-temporal features of the past video frames and generating at least one frame;
• discriminating, by an image discriminator, at least one generated frame and at least one real frame of the past video frames;
• discriminating, by a video discriminator, at least one generated video and at least one real video, wherein the at least one real video includes the past video frames and at least one future frame;
• training a fully convolutional transformer network (FCTN) model;
• and
• determining an anomaly score of at least one test video based on a prediction residual map from the FCTN model.

A computer program product for detecting anomaly in video data, the computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computing device to cause the computing device to perform the method comprising:

• receiving, at a generator, past video frames, wherein the generator includes fill r convolutional transformer based generative adversarial networks (FCT-GANs);
• extracting, by the generator, spatio-temporal features of the past video frames and generating at least one frame;
• discriminating, by an image discriminator, at least one generated frame and at least one real frame of the past video frames;
• discriminating, by a video discriminator, at least one generated video and at least one real video, wherein the at least one real video includes the past video frames and at least one future frame;
• training a fully convolutional transformer network (FCTN) model;
• and
• determining an anomaly score of at least one test video based on a prediction residual map from the model.