Methods, systems and media for evaluating layered computer security products

A system for evaluating layered security products, the system comprising:

• at least one hardware processor programmed to:
• (a) record attacks on a computing target, identify a successful one of the attacks as a known attack, and identify attack data associated with the known attack;
• (b) identify a plurality of portions of the attack data associated with the known attack, wherein at least one of the plurality of portions of the attack data is at least a portion of an executable file delivered by the known attack;
• (c) test a plurality of security products using the plurality of portions of the attack data of the known attack, wherein at least two of the plurality of security products use different portions of the plurality of portions of the attack data to detect the known attack and at least one of the different portions of the plurality of portions of the attack data is the at least a portion of the executable file;
• (d) repeat (c) for a plurality of known attacks;
• receive information identifying a subset of the plurality of security products;
• identify, for each of the subset of the plurality of security products, a first set of identified known attacks detected by the security product from the plurality of known attacks using results of the testing in (c);
• and
• determine a first number of attacks in a first union of each of the first sets of identified known attacks.

A non-transitory computer-readable medium containing computer executable instructions that, when executed by a processor, cause the processor to perform a method for evaluating layered security products, the method comprising:

• (a) recording attacks on a computing target, identifying a successful one of the attacks as a known attack, and identifying attack data associated with the known attack;
• (b) identifying a plurality of portions of the attack data associated with the known attack, wherein at least one of the plurality of portions of the attack data is at least a portion of an executable file delivered by the known attack;
• (c) testing a plurality of security products using the plurality of portions of the attack data of the known attack, wherein at least two of the plurality of security products use different portions of the plurality of portions of the attack data to detect the known attack and at least one of the different portions of the plurality of portions of the attack data is the at least a portion of the executable file;
• (d) repeating (c) for a plurality of known attacks;
• receiving information identifying a subset of the plurality of security products;
• identifying, for each of the subset of the plurality of security products, a first set of identified known attacks detected by the security product from the plurality of known attacks using results of the testing in (c);
• and
• determining a first number of attacks in a first union of each of the first sets of identified known attacks.