# US 12,184,522 B2 · Application service level expectation health and performance > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,184,522 B2 - Document ID: US12184522B2 - Status: Granted - Kind: B2 - Application number: 18/155,658 - Filed: 2023-01-17 - Published: 2023-07-20 - Granted: 2024-12-31 - Earliest claimed: 2022-01-17 - Assignees: Juniper Networks, Inc. - Applicants: Juniper Networks, Inc. - Inventors: Prashant Kumar; Jisheng Wang; Gorakhanath Kathare; Yogesh B G; Kaushik Adesh Agrawal; Jie C Jiang; Scott A. McCulley; Greg Schrock - Web page: https://patentgrep.com/patent/US12184522B2 ## CPC - H04L 41/064: {involving time analysis} - H04L 43/0817: by checking functioning - H04L 41/0627: {by acting on the notification or alarm source} - H04L 41/0631: using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis - H04L 41/069: using logs of notifications; Post-processing of notifications - H04L 41/0816: the condition being an adaptation, e.g. in response to network events - H04L 41/0886: {Fully automatic configuration} - H04L 41/122: of virtualised topologies, e.g. software-defined networks [SDN] or network function virtualisation [NFV] - H04L 41/147: for predicting network behaviour - H04L 41/16: using machine learning or artificial intelligence - H04L 43/067: using time frame reporting - H04L 41/046: comprising network management agents or mobile agents therefor - H04L 41/082: the condition being updates or upgrades of network functionality - H04L 41/142: using statistical or mathematical methods - H04L 41/145: {involving simulating, designing, planning or modelling of a network} - H04L 41/22: comprising specially adapted graphical user interfaces [GUI] - H04L 41/5009: Determining service level performance parameters or violations of service level contracts, e.g. violations of agreed response time or mean time between failures [MTBF] - H04L 41/5019: Ensuring fulfilment of SLA - H04L 41/5067: Customer-centric QoS measurements - H04L 43/06: Generation of reports - H04L 43/0805: by checking availability - H04L 43/0829: Packet loss - H04L 43/0852: Delays - H04L 43/0876: Network utilisation, e.g. volume of load or congestion level - H04L 43/16: Threshold monitoring - H04L 43/20: the monitoring system or the monitored elements being virtualised, abstracted or software-defined entities, e.g. SDN or NFV - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract Techniques are described for monitoring application performance in a computer network. For example, a network management system (NMS) includes a memory storing path data received from a plurality of network devices, the path data reported by each network device of the plurality of network devices for one or more logical paths of a physical interface from the given network device over a wide area network (WAN). Additionally, the NMS may include processing circuitry in communication with the memory and configured to: determine, based on the path data, one or more application health assessments for one or more applications, wherein the one or more application health assessments are associated with one or more application time periods for a site, and in response to determining at least one failure state, output a notification including identification of a root cause of the at least one failure state. ## Claims ### Claim 1 A network management system (NMS) of an enterprise network, the network management system comprising: a memory; and processing circuitry in communication with the memory and configured to: obtain path data of a plurality of network devices operating as network gateways for the enterprise network, wherein the path data of a first network gateway is associated with one or more logical paths of a physical interface of the first network gateway with one or more other network gateways over a wide area network (WAN); determine, based on the path data, one or more application health assessments of an application of a plurality of applications, wherein the one or more application health assessments are associated with one or more application sessions of the application during each application time period of a sequence of consecutive application time periods, wherein an application time period corresponds to a period of time during which an application server of the application in communication with the first network gateway provides service over the one or more application sessions to a client device within a site of the enterprise network via a logical path of the one or more logical paths of the first network gateway with a second network gateway for the site over the WAN, and wherein an application session of the one or more application sessions includes data flows between at least the first network gateway and the second network gateway during the application time period; determine at least one application time period in the sequence of consecutive application time periods as a failure state of the application based on the one or more application health assessments for the one or more application sessions during the application time period; and based on determining the at least one application time period as a failure state, output a notification including identification of a root cause of the failure state. ### Claim 2 (depends on 1) The NMS of claim 1, wherein the processing circuitry is configured to: determine a list of top-interest applications for the site of the enterprise network from the plurality of applications based on one or more of an application session count or an application session duration for each application of the plurality of applications at the site; and determine, based on the path data, the one or more application health assessments for each application of the list of top-interest applications for the site. ### Claim 3 (depends on 1) The NMS of claim 1, wherein to determine the at least one application time period as a failure state of the application, the processing circuitry is configured to apply one or more thresholds to classify each application time period of the plurality of in the sequence of consecutive application time periods as a high-quality application time period or a low-quality application time period. ### Claim 4 (depends on 3) The NMS of claim 3, wherein the processing circuitry is configured to determine the one or more thresholds based on the path data and historical application performance data. ### Claim 5 (depends on 3) The NMS of claim 3, wherein the processing circuitry is further configured to classify each low-quality application time period based on an identified application-related problem classifier. ### Claim 6 (depends on 5) The NMS of claim 5, wherein the application-related problem classifier comprises one of: a slow application problem classifier corresponding to slow response time from the application server; a transmission control protocol (TCP) retransmission problem classifier corresponding to one or more retries from the client device or the application server caused by service unavailability; an application bandwidth problem classifier corresponding to bandwidth being lower than a threshold amount of bandwidth required by the application; or an application disconnect problem classifier corresponding to a plurality of disconnections from the client device or the application server. ### Claim 7 (depends on 1) The NMS of claim 1, wherein the path data includes latency, jitter, and loss corresponding to each logical path of the one or more logical paths of the first network gateway with the other network gateways over the WAN. ### Claim 8 (depends on 1) The NMS of claim 1, wherein the memory is further configured to store one or more machine learning models, and wherein to determine the at least one application time period as a failure state of the application, the processing circuitry is further configured to: execute the one or more machine learning models in order to classify each application time period in the sequence of consecutive application time periods as a high-quality application time period or a low-quality application time period based on the path data. ### Claim 9 (depends on 8) The NMS of claim 8, wherein to determine the at least one application time period as a failure state of the application, the processing circuitry is further configured to: execute the one or more machine learning models in order to classify each application time period in the sequence of consecutive application time periods as a high-quality application time period or a low-quality application time period based on the path data and user feedback data corresponding to the sequence of consecutive application time periods. ### Claim 10 (depends on 8) The NMS of claim 8, wherein the processing circuitry is further configured to execute the one or more machine learning models in order to classify each low-quality application time period based on an identified application-related problem classifier. ### Claim 11 A method comprising: obtaining, by processing circuitry of a network management system (NMS) of an enterprise network, path data of a plurality of network devices operating as network gateways for the enterprise network, wherein the path data of a first network gateway is associated with one or more logical paths of a physical interface of the first network gateway with one or more other network gateways over a wide area network (WAN); determining, by the processing circuitry and based on the path data, one or more application health assessments of an application of a plurality of applications, wherein the one or more application health assessments are associated with one or more application sessions of the application during each application time period of a sequence of consecutive application time periods, wherein each an application time period corresponds to a period of time during which an application server of the application in communication with the first network gateway provides service over the one or more application sessions to a client device within a site of the enterprise network via a logical path of the one or more logical paths of the first network gateway with a second network gateway for the site over the WAN, and wherein an application session of the one or more application sessions includes data flows between at least the first network gateway and the second network gateway during the application time period; determining, by the processing circuitry, at least one application time period in the sequence of consecutive application time periods as a failure state of the application based on the one or more application health assessments for the one or more application sessions during the application time period; and based on determining the at least one application time period as a failure state, outputting, by the processing circuitry, a notification including identification of a root cause of the failure state. ### Claim 12 (depends on 11) The method of claim 11, further comprising: determining, by the processing circuitry, a list of top-interest applications for the site of the enterprise network from the plurality of applications based on one or more of an application session count or an application session duration for each application of the plurality of applications at the site; and determining, by the processing circuitry based on the path data, the one or more application health assessments for each application of the list of top-interest applications for the site. ### Claim 13 (depends on 11) The method of claim 11, wherein determining the at least one application time period as a failure state of the application comprises applying, by the processing circuitry, one or more thresholds to classify each application time period in the sequence of consecutive application time periods as a high-quality application time period or a low-quality application time period. ### Claim 14 (depends on 13) The method of claim 13, further comprising determining, by the processing circuitry, the one or more thresholds based on the path data and historical application performance data. ### Claim 15 (depends on 13) The method of claim 13, further comprising classifying, by the processing circuitry, each low-quality application time period based on an identified application-related problem classifier. ### Claim 16 (depends on 15) The method of claim 15, wherein the application-related problem classifier comprises one of: a slow application problem classifier corresponding to slow response time from the application server; a transmission control protocol (TCP) retransmission problem classifier corresponding to one or more retries from the client device or the application server caused by service unavailability; an application bandwidth problem classifier corresponding to bandwidth being lower than a threshold amount of bandwidth required by the application; or an application disconnect problem classifier corresponding to frequent a plurality of disconnections from the client device or the application server. ### Claim 17 (depends on 11) The method of claim 11 wherein determining the at least one application time period as a failure state of the application further comprises: executing, by the processing circuitry, one or more machine learning models in order to classify each application time period in the sequence of consecutive application time periods as a high-quality application time period or a low-quality application time period based on the path data. ### Claim 18 (depends on 17) The method of claim 17, wherein determining the at least one application time period as a failure state of the application further comprises: executing, by the processing circuitry, the one or more machine learning models in order to classify each application time period in the sequence of consecutive application time periods as a high-quality application time period or a low-quality application time period based on the path data and user feedback data corresponding to the sequence of consecutive application time periods. ### Claim 19 (depends on 17) The method of claim 17, further comprising executing, by the processing circuitry, the one or more machine learning models in order to classify each low-quality application time period based on an identified application-related problem classifier. ### Claim 20 Non-transitory computer-readable storage media comprising instructions for causing one or more processors of a network management system (NMS) of an enterprise network to: obtain path data of a plurality of network devices operating as network gateways for the enterprise network, wherein the path data of a first network gateway is associated with one or more logical paths of a physical interface of the first network gateway with one or more other network gateways over a wide area network (WAN); determine, based on path data, one or more application health assessments of an application of a plurality of applications, wherein the one or more application health assessments are associated with one or more application sessions of the application during each application time period of a sequence of consecutive application time periods, wherein an application time period corresponds to a period of time during which an application server of the application in communication with the first network gateway provides service over the one or more application sessions to a client device within a site of the enterprise network via a logical path of the one or more logical paths of the first network gateway with a second network gateway for the site over the WAN, and wherein an application session of the one or more application sessions includes data flows between at least the first network gateway and the second network gateway during the application time period; determine at least one application time period in the sequence of consecutive application time periods as a failure state of the application based on the one or more application health assessments for the one or more application sessions during the application time period; and based on determining the at least one application time period as a failure state, outputting a notification including identification of a root cause of the failure state.