Method and system for generating adversarial attacks on video recognition models

A method for generating adversarial attacks on a video recognition model, comprising:

• (a) generating content of bullet-screen comments (BSCs) by an image captioning model for a clean video sample;
• (b) generating an adversarial video sample by inserting the BSCs into the clean video sample;
• (c) using the adversarial video sample to attack the video recognition model;
• (d) receiving rewards from the environment;
• (e) optimizing position and transparency of the BSCs by a reinforced learning (RL) agent according to the received rewards;
• and
• (f) updating the adversarial video sample and using the adversarial video sample to attack the video recognition model;
• iteratively repeating steps of (d)-(f), until a predefined condition is matched.

A system for generating adversarial attacks on a video recognition model, comprising:

• a content generating module generating content of bullet-screen comments (BSCs) by an image captioning model for a clean video sample;
• an adversarial video sample generating module generating an adversarial video sample by inserting the BSCs into the clean video sample;
• and
• an iterative attacking module iteratively repeating steps as follows:
• attacking the video recognition model by using the adversarial video sample;
• receiving a fooling rate from the video recognition model and an Intersection over Union (IoU) between different BSCs;
• optimizing the position and transparency of the BSCs by a reinforced learning (RL) agent according to the received fooling rate and the IoU;
• and
• updating the adversarial video sample;
• wherein the iterative attacking module repeating the above steps until the IoU between the BSCs in the adversarial video sample is equal to 0.

A system for generating adversarial attacks on a video recognition model, comprising:

• a content generating module generating content of bullet-screen comments (BSCs) by an image captioning model for a clean video sample;
• an adversarial video sample generating module generating an adversarial video sample by inserting the BSCs into the clean video sample;
• an iterative attacking module iteratively repeating the following steps:
• attacking the video recognition model by using the adversarial video sample;
• receiving rewards from an environment;
• optimizing position and transparency of the BSCs by a reinforced learning (RL) agent according to the received rewards;
• updating the adversarial video sample and using the adversarial video sample to attack the video recognition model;
• wherein the iterative attacking module repeats the above steps until a predefined condition is matched.