# US 12,181,956 B1 · Machine-learning based prioritization of alert groupings > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,181,956 B1 - Document ID: US12181956B1 - Status: Granted - Kind: B1 - Application number: 18/208,879 - Filed: 2023-06-12 - Granted: 2024-12-31 - Earliest claimed: 2022-01-28 - Assignees: Splunk Inc. - Applicants: Splunk Inc. - Inventors: Kristal Curtis; William Deaderick; Wei J. Gao; Tanner Gilligan; Chandrima Sarkar; Aleksander Stojanovic; Ralph Donald Thompson; Poonam Yadav; Sichen Zhong - Web page: https://patentgrep.com/patent/US12181956B1 ## CPC - G06F 11/0781: {Error filtering or prioritizing based on a policy defined by the user or on a policy defined by a hardware/software module, e.g. according to a severity level} - G06F 11/0769: {Readable error formats, e.g. cross-platform generic formats, human understandable formats} - G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting - G06F 18/2178: {based on feedback of a supervisor} - G06F 18/24137: {Distances to cluster centroïds} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Systems and methods are disclosed that are directed to improving the prioritization, display, and viewing of system alerts through the use of machine learning techniques to group the alerts and further to prioritize the groupings. Additionally, a graphical user interface is generated that illustrates the prioritized listing of the plurality of groupings. Thus, a system administrator or other user receives an improved experience as the number of notifications provided to the system administrator are reduced due to the grouping of individual alerts into related groupings and further due to the prioritization of the groupings. Previously, or in current technology, system alerts may be automatically generated and provided immediately to a system administrator. In some instances, any advantage of detecting system errors or system monitoring provided by the alerts is negated by the vast number of alerts and provision of minimally important alerts in a manner that concealed more important alerts. ## Claims ### Claim 1 A computerized method comprising: receiving a plurality of alerts, wherein the plurality of alerts are grouped into a plurality of groupings, wherein one or more of the plurality of groupings include two or more alerts of the plurality of alerts; determining a feature vector of each grouping of the plurality of groupings; generating a score for each grouping of the plurality of groupings through evaluation of the feature vector of each of the plurality of groupings by a machine learning model, wherein the score corresponds to a prediction of a priority of the plurality of alerts associated with each of the plurality of groupings; generating a prioritized listing of the plurality of groupings based on the score generated for each of the plurality of groupings; and generating a graphical user interface (GUI) configured for rendering on a network device of a user, wherein the GUI illustrates the prioritized listing of the plurality of groupings. ### Claim 2 (depends on 1) The computerized method of claim 1, wherein alerts comprising a first grouping of the plurality of groupings correspond to a common incident. ### Claim 3 (depends on 1) The computerized method of claim 1, wherein each of the plurality of alerts include a same set of fields, and wherein the feature vector of a first grouping is a statistical aggregation of the feature vector of each alerts comprising the first grouping. ### Claim 4 (depends on 1) The computerized method of claim 1, wherein a first field of a first alert represents a first feature of the feature vector. ### Claim 5 (depends on 1) The computerized method of claim 1, further comprising: receiving user feedback on the prioritized listing of the plurality of groupings, wherein the user feedback is either implicit user feedback or explicit user feedback; retraining the machine learning model based on the user feedback; and evaluating subsequently received alerts with the retrained machine learning model. ### Claim 6 (depends on 5) The computerized method of claim 5, wherein the explicit user feedback indicates either approval or disapproval of placement of an alert within the prioritized listing, and wherein the implicit user feedback is derived from telemetry data collected based on the user viewing the prioritized listing, actions taken by the user in response to viewing the prioritized listing, or actions not taken by the user in response to viewing the prioritized listing. ### Claim 7 (depends on 1) The computerized method of claim 1, further comprising: prior to generating the score for each grouping of the plurality of groupings, training the machine learning model including: providing a multi-layer neural network with an alert grouping pair from a training data including a label, wherein forward propagation is performed for both alert grouping producing an output value for each alert grouping, and determining a difference between the output values for each alert grouping and provided to a sigmoid node, where a cross entropy loss between a sigmoid and the label is back-propagated into the multi-layer neural network resulting in computation of a model gradient and updating a weighting scheme. ### Claim 8 A computing device, comprising: one or more processors; and a non-transitory computer-readable medium having stored thereon instructions that, when executed by the processor, cause the processor to perform operations including: receiving a plurality of alerts, wherein the plurality of alerts are grouped into a plurality of groupings, wherein one or more of the plurality of groupings include two or more alerts of the plurality of alerts; determining a feature vector of each grouping of the plurality of groupings; generating a score for each grouping of the plurality of groupings through evaluation of the feature vector of each of the plurality of groupings by a machine learning model, wherein the score corresponds to a prediction of a priority of the plurality of alerts associated with each of the plurality of groupings; generating a prioritized listing of the plurality of groupings based on the score generated for each of the plurality of groupings; and generating a graphical user interface (GUI) configured for rendering on a network device of a user, wherein the GUI illustrates the prioritized listing of the plurality of groupings. ### Claim 9 (depends on 8) The computing device of claim 8, wherein alerts comprising a first grouping of the plurality of groupings correspond to a common incident. ### Claim 10 (depends on 8) The computing device of claim 8, wherein each of the plurality of alerts include a same set of fields, and wherein the feature vector of a first grouping is a statistical aggregation of the feature vector of each alerts comprising the first grouping. ### Claim 11 (depends on 8) The computing device of claim 8, wherein a first field of a first alert represents a first feature of the feature vector. ### Claim 12 (depends on 8) The computing device of claim 8, wherein the operations further include: receiving user feedback on the prioritized listing of the plurality of groupings, wherein the user feedback is either implicit user feedback or explicit user feedback; retraining the machine learning model based on the user feedback; and evaluating subsequently received alerts with the retrained machine learning model. ### Claim 13 (depends on 12) The computing device of claim 12, wherein the explicit user feedback indicates either approval or disapproval of placement of an alert within the prioritized listing, and wherein the implicit user feedback is derived from telemetry data collected based on the user viewing the prioritized listing, actions taken by the user in response to viewing the prioritized listing, or actions not taken by the user in response to viewing the prioritized listing. ### Claim 14 (depends on 8) The computing device of claim 8, wherein the operations further include: prior to generating the score for each grouping of the plurality of groupings, training the machine learning model including: providing a multi-layer neural network with an alert grouping pair from a training data including a label, wherein forward propagation is performed for both alert grouping producing an output value for each alert grouping, and determining a difference between the output values for each alert grouping and provided to a sigmoid node, where a cross entropy loss between a sigmoid and the label is back-propagated into the multi-layer neural network resulting in computation of a model gradient and updating a weighting scheme. ### Claim 15 A non-transitory storage medium having stored thereon instructions that, when executed, cause performance of operations including: receiving a plurality of alerts, wherein the plurality of alerts are grouped into a plurality of groupings, wherein one or more of the plurality of groupings include two or more alerts of the plurality of alerts; determining a feature vector of each grouping of the plurality of groupings; generating a score for each grouping of the plurality of groupings through evaluation of the feature vector of each of the plurality of groupings by a machine learning model, wherein the score corresponds to a prediction of a priority of the plurality of alerts associated with each of the plurality of groupings; generating a prioritized listing of the plurality of groupings based on the score generated for each of the plurality of groupings; and generating a graphical user interface (GUI) configured for rendering on a network device of a user, wherein the GUI illustrates the prioritized listing of the plurality of groupings. ### Claim 16 (depends on 15) The non-transitory storage medium of claim 15, wherein alerts comprising a first grouping of the plurality of groupings correspond to a common incident. ### Claim 17 (depends on 15) The non-transitory storage medium of claim 15, wherein each of the plurality of alerts include a same set of fields, and wherein the feature vector of a first grouping is a statistical aggregation of the feature vector of each alerts comprising the first grouping. ### Claim 18 (depends on 15) The non-transitory storage medium of claim 15, wherein a first field of a first alert represents a first feature of the feature vector. ### Claim 19 (depends on 15) The non-transitory storage medium of claim 15, wherein the operations further include: receiving user feedback on the prioritized listing of the plurality of groupings, wherein the user feedback is either implicit user feedback or explicit user feedback; retraining the machine learning model based on the user feedback; and evaluating subsequently received alerts with the retrained machine learning model, and wherein the explicit user feedback indicates either approval or disapproval of placement of an alert within the prioritized listing, and wherein the implicit user feedback is derived from telemetry data collected based on the user viewing the prioritized listing, actions taken by the user in response to viewing the prioritized listing, or actions not taken by the user in response to viewing the prioritized listing. ### Claim 20 (depends on 15) The non-transitory storage medium of claim 15, wherein the operations further include: prior to generating the score for each grouping of the plurality of groupings, training the machine learning model including: providing a multi-layer neural network with an alert grouping pair from a training data including a label, wherein forward propagation is performed for both alert grouping producing an output value for each alert grouping, and determining a difference between the output values for each alert grouping and provided to a sigmoid node, where a cross entropy loss between a sigmoid and the label is back-propagated into the multi-layer neural network resulting in computation of a model gradient and updating a weighting scheme.