# US 11,909,773 B2 · Continuous improvement of security detection rules > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,909,773 B2 - Document ID: US11909773B2 - Status: Granted - Kind: B2 - Application number: 17/587,303 - Filed: 2022-01-28 - Published: 2022-10-06 - Granted: 2024-02-20 - Earliest claimed: 2021-04-02 - Assignees: Target Brands, Inc. - Applicants: Target Brands, Inc. - Inventors: Paul Hutelmyer; Adam Blake - Web page: https://patentgrep.com/patent/US11909773B2 ## CPC - H04L 63/1433: {Vulnerability analysis} - H04L 63/205: {involving negotiation or determination of the one or more network security mechanisms to be used, e.g. by negotiation between the client and the server or between peers or by selection according to the capabilities of the entities involved (negotiation of communication capabilities H04L69/24)} - H04L 63/20: {for managing network security; network security policies in general (filtering policies H04L63/0227)} - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract Systems and methods disclosed can evaluate security detection rules in a network security computing environment. Results for a processed log of security events can be retrieved. The results can identify determined outcomes for instances triggering security detection rules. The security detection rules can detect specific behavior on a network by being processed against a log of security events. Scores for the security detection rules can be determined based on the results of the processed log of security events and the determined outcomes. The security detection rules can be ranked based on the scores, from highest to lowest score. The highest score can indicate that a corresponding rule is performing worst among the security detection rules and the lowest score can indicate that a corresponding rule is performing best among the security detection rules. A rules score report can be generated based on the ranked rules. ## Claims ### Claim 1 A system for evaluating security detection rules in a network security computing environment, the system comprising: a network security system configured to (i) receive a log of security events for a network and (ii) process the log of security events against a plurality of security detection rules, wherein each of the plurality of security detection rules is configured to detect a specific network behavior on the network through being processed against the log of security events; a log results data store configured to: receive, from the network security system, results for the processed log of security events against the plurality of security detection rules; and store the results for the processed log of security events; a detection hub system for assessing the plurality of security detection rules that were triggered by processing of the log of security events, the detection hub system configured to: retrieve, from the log results data store, the results for the processed log of security events, wherein the results identify determined outcomes for instances triggering the plurality of security detection rules; determine scores for the plurality of security detection rules based on the results of the processed log of security events and the determined outcomes, wherein the detection hub system is configured to determine scores for the plurality of security detection rules by: determining, for each rule of the plurality of security detection rules, a number of times that the rule was properly triggered; assigning a low score based on the number of times that the rule was properly triggered exceeding a first threshold value; and assigning a high score based on the number of times that the rule was properly triggered being less than the first threshold value; rank the plurality of security detection rules based on the scores, wherein the plurality of security detection rules are ranked from a highest score to a lowest score, wherein the highest score indicates that a corresponding rule is performing worst among the plurality of security detection rules and the lowest score indicates that a corresponding rule is performing best among the plurality of security detection rules; and generate, based on the ranked rules, a rules score report; and a user computing device having a graphical user interface (GUI) display, the user computing device configured to: receive, from the detection hub system, the rules score report; output, for display at the GUI display, the rules score report; modify, based on the rules score report, one or more of the rules identified in the rules score report; and store, in a rules data store, the modified one or more rules. ### Claim 2 (depends on 1) The system of claim 1, wherein the detection hub system is further configured to adjust the scores for the plurality of security detection rules, wherein adjusting the scores comprises (i) decreasing one or more scores based on one or more corresponding rules being triggered more than a second threshold value and (ii) increasing the one or more scores based on the one or more corresponding rules being triggered less than the second threshold value. ### Claim 3 (depends on 2) The system of claim 2, wherein adjusting the scores for the plurality of security detection rules further comprises (i) increasing one or more scores based on a quantity of cases that trigger one or more corresponding rules exceeding a third threshold value and (ii) decreasing the one or more scores based on the quantity of cases that trigger the one or more corresponding rules being less than the third threshold value. ### Claim 4 (depends on 2) The system of claim 2, wherein adjusting the scores for the plurality of security detection rules further comprises (i) increasing one or more scores based on a type of one or more corresponding rules being associated with high risk threats and (ii) decreasing the one or more scores based on the type of the one or more corresponding rules being associated with non-high risk threats. ### Claim 5 (depends on 2) The system of claim 2, wherein adjusting the scores for the plurality of security detection rules further comprises (i) decreasing one or more scores based on modifications being made to one or more corresponding rules in less than a predetermined timeframe and (ii) increasing the one or more scores based on the modifications being made to the one or more corresponding rules in greater than the predetermined timeframe. ### Claim 6 (depends on 1) The system of claim 1, wherein the detection hub system is further configured to store, in the rules data store, at least one of (i) the scores for the plurality of security detection rules, (ii) the ranked rules, and (iii) the rules score report. ### Claim 7 (depends on 1) The system of claim 1, wherein the detection hub system is further configured to: adjust the scores for the plurality of security detection rules relative to scores of each nearest neighbor of a respective rule, wherein ranking the plurality of security detection rules based on the scores of each nearest neighbor of the respective rule comprises: generating an initial ranking of the plurality of security detection rules based on the highest score to the lowest score; applying a use case to the respective rule; decreasing, in response to determining that the respective rule satisfies the use case, a score of the respective rule relative to scores of each nearest neighbor of the respective rule in the initially ranked rules; increasing, in response to determining that the respective rule does not satisfy the use case, the score of the respective rule relative to the scores of each nearest neighbor of the respective rule in the initially ranked rules; and generating a finalized ranking of the rules. ### Claim 8 A system for evaluating security detection rules in a network security computing environment, the system comprising: a network security system configured to (i) receive a log of security events for a network and (ii) process the log of security events against a plurality of security detection rules, wherein each of the plurality of security detection rules is configured to detect a specific network behavior on the network through being processed against the log of security events; a log results data store configured to: receive, from the network security system, results for the processed log of security events against the plurality of security detection rules; and store the results for the processed log of security events; a detection hub system for assessing the plurality of security detection rules that were triggered by processing of the log of security events, the detection hub system configured to: retrieve, from the log results data store, the results for the processed log of security events, wherein the results identify determined outcomes for instances triggering the plurality of security detection rules; determine scores for the plurality of security detection rules based on the results of the processed log of security events and the determined outcomes; adjust the scores for the plurality of security detection rules relative to scores of each nearest neighbor of a respective rule; rank the plurality of security detection rules based on the scores, wherein the plurality of security detection rules are ranked from a highest score to a lowest score, wherein the highest score indicates that a corresponding rule is performing worst among the plurality of security detection rules and the lowest score indicates that a corresponding rule is performing best among the plurality of security detection rules, wherein ranking the plurality of security detection rules based on the scores of each nearest neighbor of the respective rule comprises: generating an initial ranking of the plurality of security detection rules based on the highest score to the lowest score; applying a use case to the respective rule; decreasing, in response to determining that the respective rule satisfies the use case, a score of the respective rule relative to scores of each nearest neighbor of the respective rule in the initially ranked rules; increasing, in response to determining that the respective rule does not satisfy the use case, the score of the respective rule relative to the scores of each nearest neighbor of the respective rule in the initially ranked rules; and generating a finalized ranking of the rules; and generate, based on the finalized ranked rules, a rules score report; and a user computing device having a graphical user interface (GUI) display, the user computing device configured to: receive, from the detection hub system, the rules score report; output, for display at the GUI display, the rules score report; modify, based on the rules score report, one or more of the rules identified in the rules score report; and store, in a rules data store, the modified one or more rules. ### Claim 9 (depends on 8) The system of claim 8, wherein the respective rule satisfies the use case when the respective rule is triggered less than a predefined threshold value. ### Claim 10 (depends on 9) The system of claim 9, wherein the respective rule does not satisfy the use case when the respective rule is triggered more than the predefined threshold value. ### Claim 11 (depends on 8) The system of claim 8, wherein the respective rule satisfies the use case when the respective rule is identified as a non-malicious event. ### Claim 12 (depends on 11) The system of claim 11, wherein the respective rule does not satisfy the use case when the respective rule is identified as a malicious event. ### Claim 13 (depends on 8) The system of claim 8, wherein the respective rule satisfies the use case when the respective rule has been modified within a predetermined timeframe. ### Claim 14 (depends on 13) The system of claim 13, wherein the respective rule does not satisfy the use case when the respective rule has not been modified with the predetermined timeframe. ### Claim 15 (depends on 8) The system of claim 8, wherein decreasing the score of the respective rule relative to the scores of each nearest neighbor of the respective rule comprises subtracting a predetermined amount from the score of the respective rule such that the score of the respective rule is less than at least one score of at least on nearest neighbor of the respective rule. ### Claim 16 (depends on 8) The system of claim 8, wherein increasing the score of the respective rule relative to the scores of each nearest neighbor of the respective rule comprises adding a predetermined amount to the score of the respective rule such that the score of the respective rule is greater than at least one score of at least one nearest neighbor of the respective rule. ### Claim 17 A method for evaluating security detection rules in a network security computing environment, the method comprising: retrieving, by a computing system, results for a processed log of security events, wherein the results identify determined outcomes for instances triggering a plurality of security detection rules, wherein each of the plurality of security detection rules is configured to detect a specific network behavior on a network through being processed against a log of security events; determining, by the computing system, scores for the plurality of security detection rules based on the results of the processed log of security events and the determined outcomes, wherein determining scores for the plurality of security detection rules comprises: determining, for each rule of the plurality of security detection rules, a number of times that the rule was properly triggered; assigning a low score based on the number of times that the rule was properly triggered exceeding a first threshold value; and assigning a high score based on the number of times that the rule was properly triggered being less than the first threshold value; ranking, by the computing system, the plurality of security detection rules based on the scores, wherein the plurality of security detection rules are ranked from a highest score to a lowest score, wherein the highest score indicates that a corresponding rule is performing worst among the plurality of security detection rules and the lowest score indicates that a corresponding rule is performing best among the plurality of security detection rules; and generating, by the computing system, a rules score report based on the ranked rules. ### Claim 18 (depends on 17) The method of claim 17, further comprising adjusting the scores for the plurality of security detection rules, wherein adjusting the scores comprises (i) decreasing one or more scores based on one or more corresponding rules being triggered more than a second threshold value and (ii) increasing the one or more scores based on the one or more corresponding rules being triggered less than the second threshold value. ### Claim 19 (depends on 18) The method of claim 18, wherein adjusting the scores for the plurality of security detection rules further comprises (i) increasing one or more scores based on a quantity of cases that trigger one or more corresponding rules exceeding a third threshold value and (ii) decreasing the one or more scores based on the quantity of cases that trigger the one or more corresponding rules being less than the third threshold value. ### Claim 20 (depends on 17) The method of claim 17, further comprising: storing, by the computing system and in a rules data store, the modified one or more rules.