# US 12,182,255 B2 · Systems and methods for automated rule-based detection > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,182,255 B2 - Document ID: US12182255B2 - Status: Granted - Kind: B2 - Application number: 17/589,773 - Filed: 2022-01-31 - Published: 2023-08-03 - Granted: 2024-12-31 - Earliest claimed: 2022-01-31 - Assignees: Walmart Apollo, LLC - Applicants: Walmart Apollo, LLC - Inventors: Siyuan Zhou; Jing Xia; Alexander Chobanyan; Bradley Lamarr Leonard - Web page: https://patentgrep.com/patent/US12182255B2 ## CPC - G06F 21/55: Detecting local intrusion or implementing counter-measures - G06F 2221/034: Test or assess a computer or a system - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Systems and methods for generating, selecting, and implementing rule-based strategies are disclosed. An input data set representing a plurality of interactions that may be classified as malicious or non-malicious is received and at least one strategy tree including a plurality of rule-based strategies is generated. The at least one strategy trees is generated by a machine learning model configured to generate a tree structure. The rule-based strategies are ranked based on a precision-recall-stability (PRS) score generated for each of the rule-based strategies and at least a first rule-based strategy having a highest PRS score is extracted. One or more interactions are evaluated using the first rule-based strategy to determine when the one or more interactions are malicious. ## Claims ### Claim 1 A system, comprising: a non-transitory memory having instructions stored thereon and a processor configured to read the instructions to: receive an input data set representing a plurality of interactions that may be classified as malicious or non-malicious; generate at least one strategy tree including a plurality of rule-based strategies, wherein the at least one strategy tree comprises a strategy tree having a highest area-under-the-curve metric among a plurality of strategy trees for a true positive verse false positive curve and trees is generated by a machine learning model configured to generate a tree structure; rank the rule-based strategies based on a precision-recall-stability (PRS) score generated for each of the rule-based strategies, wherein the PRS score is a single metric combining Precision, Recall and Stability of each rule-based strategy; extract at least a first rule-based strategy having a highest PRS score; and evaluate one or more interactions using the first rule-based strategy to determine when the one or more interactions are malicious. ### Claim 2 (depends on 1) The system of claim 1, wherein the PRS score is determined according to: PRS = 3 1 α * Precision + 1 β * Recall + 1 γ * Stability where each of α, β, and γ are independent weighting constants. ### Claim 3 (depends on 2) The system of claim 2, wherein Precision is a ratio of true positive identifications to a total number of positive identifications of a selected rule-based strategy. ### Claim 4 (depends on 2) The system of claim 2, wherein Recall is a ratio of true positive identifications to a total number of malicious interactions in an input data set. ### Claim 5 (depends on 2) The system of claim 2, wherein Stability is determined based on variations in Precision and Recall over time. ### Claim 6 (depends on 1) The system of claim 1, wherein the first rule-based strategy is implemented in a real-time decision engine language. ### Claim 7 A non-transitory computer readable medium having instructions stored thereon, wherein the instructions, when executed by a processor cause a device to perform operations comprising: receiving an input data set representing a plurality of interactions that may be classified as malicious or non-malicious; generating at least one strategy tree including a plurality of rule-based strategies, wherein the at least one strategy tree comprises a strategy tree having a highest area-under-the-curve metric among a plurality of strategy trees for a true positive verse false positive curve and is generated by a machine learning model configured to generate a tree structure; ranking the rule-based strategies based on a precision-recall-stability (PRS) score generated for each of the rule-based strategies, wherein the PRS score is a single metric combining Precision, Recall and Stability of each rule-based strategy; extracting at least a first rule-based strategy having a highest PRS score; and evaluating one or more interactions using the first rule-based strategy to determine when the one or more interactions are malicious. ### Claim 8 (depends on 7) The non-transitory computer readable medium of claim 7, wherein the PRS score is determined according to: PRS = 3 1 α * Precision + 1 β * Recall + 1 γ * Stability where each of α, β, and γ are independent weighting constants. ### Claim 9 (depends on 8) The non-transitory computer readable medium of claim 8, wherein Precision is a ratio of true positive identifications to a total number of positive identifications of a selected rule-based strategy. ### Claim 10 (depends on 8) The non-transitory computer readable medium of claim 8, wherein Recall is a ratio of true positive identifications to a total number of malicious interactions in an input data set. ### Claim 11 (depends on 8) The non-transitory computer readable medium of claim 8, wherein Stability is determined based on variations in Precision and Recall over time. ### Claim 12 (depends on 7) The non-transitory computer readable medium of claim 7, wherein the first rule-based strategy is implemented in a real-time decision engine language. ### Claim 13 A method, comprising: receiving an input data set representing a plurality of interactions that may be classified as malicious or non-malicious; generating at least one strategy tree including a plurality of rule-based strategies, wherein the at least one strategy tree comprises a strategy tree having a highest area-under-the-curve metric among a plurality of strategy trees for a true positive verse fake positive curve and is generated by a machine learning model configured to generate a tree structure; ranking the rule-based strategies based on a precision-recall-stability (PRS) score generated for each of the rule-based strategies, wherein the PRS score is a single metric combining Precision, Recall and Stability of each rule-based strategy; extracting at least a first rule-based strategy having a highest PRS score; and evaluating one or more interactions using the first rule-based strategy to determine when the one or more interactions are malicious. ### Claim 14 (depends on 13) The method of claim 13, wherein the PRS score is determined according to: PRS = 3 1 α * Precision + 1 β * Recall + 1 γ * Stability where each of α, β, and γ are independent weighting constants. ### Claim 15 (depends on 14) The method of claim 14, wherein Precision is a ratio of true positive identifications to a total number of positive identifications of a selected rule-based strategy, Recall is a ratio of true positive identifications to a total number of malicious interactions in an input data set, and Stability is determined based on variations in Precision and Recall over time.