System and method for detecting malicious messages generated by a large language model (LLM)
AssigneeCisco Technology, Inc.
InventorMichael Roytman
Life of this patent
Estimating the term…
Abstract
A system and method are provided for detecting malicious messages using a two-step Bayesian approach. A discrimination engine determines for each of the messages a first score and a second score. The first score represents a likelihood that the respective messages are malicious messages, and the second score represents a likelihood that they were generated by a machine learning (ML) method, such as a large language model (LLM). Using a combination of these two scores, message with a high probability of being malicious message are discriminated and marked as such. For example, messages for which the first and second scores exceed respective thresholds are marked as suspicious.
Claims16
A method of detecting malicious messages, the method comprising:
- determining, using a discrimination engine, a first score for a plurality of messages, wherein, for a respective message, a corresponding value of the first score represents a likelihood that the respective messages is a malicious message;
- determining a first subset of the plurality of messages for which the first score of each respective message exceeds a first threshold;
- determining, using the discrimination engine, a second score for the first subset of messages of the plurality of messages, wherein, for the respective message, a corresponding value of the second score represents a likelihood that the respective message has been generated using one or more machine learning (ML) methods;
- determining a second subset of the plurality of messages for which the second score of each respective message in the first subset exceeds a second threshold;
- signaling that the second subset are suspected to be malicious messages;
- determining a third subset of the plurality of messages for which the first score exceeds a third threshold that is greater than the first threshold, wherein the third subset is determined without comparing the second score to the second threshold;
- and
- including the third subset in the one or more messages that are signaled as being suspected to be malicious messages.
A computing apparatus comprising:
- a processor;
- and
- a memory storing instructions that, when executed by the processor, configure the apparatus to:
- determine a first score for a plurality of messages for a plurality of messages, a discrimination engine of the apparatus being configured to determine the first score, wherein, for a respective message, a corresponding value of the first score represents a likelihood that the respective messages is a malicious message;
- determine a first subset of the plurality of messages for which the first score of each respective message exceeds a first threshold;
- use the discrimination engine to determine a second score for the first subset of messages of the plurality of messages, wherein, for the respective message, a corresponding value of the second score represents a likelihood that the respective message has been generated using one or more machine learning (ML) methods;
- determine a second subset of the plurality of messages for which the second score of each respective message in the first subset exceeds a second threshold;
- signal that the second subset are suspected to be malicious messages;
- determine a third subset of the plurality of messages for which the first score exceeds a third threshold that is greater than the first threshold, wherein the third subset is determined without comparing the second score to the second threshold;
- and
- include the third subset in the one or more messages that are signaled as being suspected to be malicious messages.
Dependencies are inferred from the claim text.