Method for training an autonomous agent using a reinforcement learning algorithm, associated autonomous agent

A non-transitory computer-readable recording medium on which a computer program is recorded comprising instructions which when executed by a processor of an autonomous agent configure the autonomous agent to implement a method for training an autonomous agent implementing a reinforcement learning algorithm to improve the performance of anti-malware software, the method comprising:

• selecting an action aimed at modifying the content of a malware;
• transmitting the selected action to an environment implementing said anti-malware software;
• receiving, from the environment, a reward representative of a probability that the malware modified by application of the selected action is considered benign by the anti-malware software, the reward being defined as:
• r
• ⁡
• (
• t
• +
• 1
• )
• =
• {
• R
• ⊂
• R
• +
• if
• ⁢
• p
• ⁡
• (
• t
• +
• 1
• )
• <
• T
• p
• ⁡
• (
• t
• +
• 1
• )
• -
• p
• ⁡
• (
• t
• )
• otherwise
• with R a positive real value, T a threshold value specific to the anti-malware software, p(t+1) a detectability score representative of a probability that the malware modified by application of the selected action is considered malicious by the anti-malware software, and p(t) the detectability score by application of an action selected in a previous iteration;
• obtaining a state representative of the malware modified by application of the selected action;
• wherein selecting an action, receiving a reward and obtaining a state are iterated as long as a stopping criterion is not reached;
• and,
• determining, by the reinforcement learning algorithm and based on the obtained rewards, a function which associates with each state at least one action to be executed, so as to maximize a sum of the obtained rewards.

An evaluation method for evaluating detectability of a malware by an environment implementing at least one anti-malware software, the method comprising:

• receiving, from an autonomous agent implementing a reinforcement learning algorithm, an action aimed at modifying content of the malware;
• modifying the content of the malware by application of said action, so as to obtain a modified malware;
• analyzing, by the anti-malware software, the modified malware;
• and transmitting, to the autonomous agent, a reward representative of a probability that the malware modified by application of the selected action (a(t)) is considered benign by the anti-malware software, the reward being defined as:
• r
• ⁡
• (
• t
• +
• 1
• )
• =
• {
• R
• ⊂
• R
• +
• if
• ⁢
• p
• ⁡
• (
• t
• +
• 1
• )
• <
• T
• p
• ⁡
• (
• t
• +
• 1
• )
• -
• p
• ⁡
• (
• t
• )
• otherwise
• with R a positive real value, T a threshold value specific to the anti-malware software, p(t+1) a detectability score representative of a probability that the malware modified by application of the selected action is considered malicious by the anti-malware software, and p(t) the detectability score by application of an action selected in a previous iteration.

A non-transitory computer-readable recording medium on which a computer program is recorded comprising instructions which when executed by a processor of an environment configure the environment to implement a method for evaluating detectability of a malware by an environment implementing at least one anti-malware software, the method comprising:

• receiving, from an autonomous agent implementing a reinforcement learning algorithm, an action aimed at modifying the content of the malware;
• modifying the content of the malware by application of said action, so as to obtain a modified malware;
• analyzing, by the anti-malware software, the modified malware;
• and transmitting, to the autonomous agent, a reward representative of a probability that the malware modified by application of the selected action (a(t)) is considered benign by the anti-malware software, the reward being defined as:
• r
• ⁡
• (
• t
• +
• 1
• )
• =
• {
• R
• ⊂
• R
• +
• if
• ⁢
• p
• ⁡
• (
• t
• +
• 1
• )
• <
• T
• p
• ⁡
• (
• t
• +
• 1
• )
• -
• p
• ⁡
• (
• t
• )
• otherwise
• with R a positive real value, T a threshold value specific to the anti-malware software, p(t+1) a detectability score representative of a probability that the malware modified by application of the selected action is considered malicious by the anti-malware software, and p(t) the detectability score by application of an action selected in a previous iteration.

An autonomous agent implementing a reinforcement learning algorithm to improve performance of anti-malware software, the agent comprising:

• at least one processor;
• and
• at least one non-transitory computer readable medium comprising instructions stored thereon which when executed by the at least one processor configure the agent to implement a method for training, comprising:
• selecting an action aimed at modifying content of a malware;
• transmitting the selected action to an environment implementing said anti-malware software;
• receiving, from the environment, a reward representative of a probability that the malware modified by application of the selected action (a(t)) is considered benign by the anti-malware software, the reward being defined as:
• r
• ⁡
• (
• t
• +
• 1
• )
• =
• {
• R
• ⊂
• R
• +
• if
• ⁢
• p
• ⁡
• (
• t
• +
• 1
• )
• <
• T
• p
• ⁡
• (
• t
• +
• 1
• )
• -
• p
• ⁡
• (
• t
• )
• otherwise
• with R a positive real value, T a threshold value specific to the anti-malware software, p(t+1) a detectability score representative of a probability that the malware modified by application of the selected action is considered malicious by the anti-malware software, and p(t) the detectability score by application of an action selected in a previous iteration;
• obtaining a state representative of the malware modified by application of the selected action;
• wherein selecting an action, receiving a reward and obtaining a state are iterated as long as a stopping criterion is not reached;
• and,
• determining, by the reinforcement learning algorithm and based on the obtained rewards, a function which associates with each state at least one action to be executed, so as to maximize a sum of the obtained rewards.

An environment for evaluating detectability of a malware by anti-malware software, the environment comprising:

• at least one processor;
• and
• at least one non-transitory computer readable medium comprising instructions stored thereon which when executed by the at least one processor configure the environment to implement, the method comprising:
• receiving, from an autonomous agent implementing a reinforcement learning algorithm, an action aimed at modifying content of the malware;
• modifying the content of the malware by application of said action, so as to obtain a modified malware;
• analyzing, by the anti-malware software, the modified malware;
• and transmitting, to the autonomous agent, a reward representative of a probability that the malware modified by application of the selected action (a(t)) is considered benign by the anti-malware software, the reward being defined as:
• r
• ⁡
• (
• t
• +
• 1
• )
• =
• {
• R
• ⊂
• R
• +
• if
• ⁢
• p
• ⁡
• (
• t
• +
• 1
• )
• <
• T
• p
• ⁡
• (
• t
• +
• 1
• )
• -
• p
• ⁡
• (
• t
• )
• otherwise
• with R a positive real value, T a threshold value specific to the anti-malware software, p(t+1) a detectability score representative of a probability that the malware modified by application of the selected action is considered malicious by the anti-malware software, and p(t) the detectability score by application of an action selected in a previous iteration.