# US 10,374,924 B1 · Virtualized network device failure detection > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,374,924 B1 - Document ID: US10374924B1 - Status: Granted - Kind: B1 - Application number: 14/562,537 - Filed: 2014-12-05 - Granted: 2019-08-06 - Earliest claimed: 2014-12-05 - Assignees: Amazon Technologies, Inc. - Applicants: Amazon Technologies, Inc. - Inventors: Ryan Christopher Holland; Thomas Charles Stickle - Web page: https://patentgrep.com/patent/US10374924B1 ## CPC - H04L 43/0817: by checking functioning - H04L 43/0823: Errors, e.g. transmission errors - H04L 41/0654: using network fault recovery (ring fault isolation or reconfiguration in loop networks without recovery actions by a network management system H04L12/437) - H04L 41/142: using statistical or mathematical methods - H04L 41/40: using virtualisation of network functions or resources, e.g. SDN or NFV entities - H04L 43/065: related to network devices - H04L 43/20: the monitoring system or the monitored elements being virtualised, abstracted or software-defined entities, e.g. SDN or NFV - H04L 41/06: Management of faults, events, alarms or notifications - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract A technology is described for detecting a failure of a primary virtualized device and failing over to a secondary virtualized network device. An example method may include generating a statistical model using historical data for network traffic that has been handled by a primary virtualized device. The statistical model may contain a functional state baseline representing a functional state of the primary virtualized device handling network traffic during a time frame. Having generated the statistical model, the functional state of the primary virtualized device may be monitored by calculating a network traffic metric for current network traffic handled by the primary virtualized device and comparing the network traffic metric for current network traffic handled by the primary virtualized device with a data point in the statistical model corresponding to the network traffic metric and the time frame for the network traffic metric. ## Claims ### Claim 1 A non-transitory machine readable storage medium having instructions embodied thereon, wherein the instructions, when executed by at least one processor: monitor a functional state of a primary virtualized network device using a hypervisor configured to: calculate TCP (Transmission Control Protocol) traffic metrics for current TCP network traffic transmitted through a network layer of the hypervisor and handled by the primary virtualized network device; compare the TCP traffic metrics for current TCP network traffic handled by the primary virtualized network device with data points in a statistical model corresponding to the TCP traffic metrics and the time frame for the TCP traffic metrics, wherein the statistical model is generated using historical data relating to previous TCP network traffic handled by the primary virtualized network device during a time frame, the statistical model containing a functional state baseline representing a functional state of the primary virtualized network device handling the previous TCP network traffic during the time frame; determine that the primary virtualized network device is in a non-functional state as a result of at least one of the data points in the statistical model corresponding to the TCP traffic metrics falling outside of the functional state baseline; and failover to a secondary virtualized network device at least partly based on having detected that the primary virtualized network device is in the non-functional state. ### Claim 2 (depends on 1) The non-transitory machine readable storage medium as in claim 1, wherein the failover is triggered as a result of determining that the primary virtualized network device is in the non-functional state. ### Claim 3 (depends on 1) The non-transitory machine readable storage medium as in claim 1, wherein the instructions, when executed by the at least one processor, cause the failover to the secondary virtualized network device by: disassociating an IP (Internet Protocol) address from the primary virtualized network device; and associating the IP address with the secondary virtualized network device. ### Claim 4 A computer implemented method, comprising: identifying a statistical model using historical data and at least one processor, the historical data being for network traffic handled by a primary virtualized device during a time frame and the statistical model containing a functional state baseline representing a functional state of the primary virtualized device handling the network traffic during the time frame; and monitoring the functional state of the primary virtualized device using a hypervisor configured for: calculating a network traffic metric for current network traffic transmitted through a network layer of the hypervisor and handled by the primary virtualized device, using the at least one processor; comparing the network traffic metric for current network traffic handled by the primary virtualized device with a data point in the statistical model corresponding to the network traffic metric and the time frame for the network traffic metric in order to monitor the functional state of the primary virtualized device using the hypervisor and the at least one processor; and determining, based on a result of the comparing, that the primary virtualized device is in a non-functional state, using the at least one processor. ### Claim 5 (depends on 4) The method as in claim 4, further comprising failing over to a secondary virtualized device at least partly based on having detected that the primary virtualized device is in the non-functional state, wherein the primary virtualized device and the secondary virtualized device are configured as a redundant pair. ### Claim 6 (depends on 4) The method as in claim 4, wherein monitoring the functional state of the primary virtualized device further comprises: collecting network traffic data for network packets transmitted through a network layer of a hypervisor, where the network packets are handled by the primary virtualized device; generating the network traffic metric using the network traffic data; identifying a time frame for the network traffic metric; and identifying the data point included in the statistical model that corresponds to the time frame. ### Claim 7 (depends on 4) The method as in claim 4, wherein monitoring the functional state of the primary virtualized device comprises collecting network traffic information associated with a state of the network traffic handled by the primary virtualized device and does not comprise obtaining state information from the primary virtualized device. ### Claim 8 (depends on 4) The method as in claim 4, wherein generating the statistical model for the network traffic that has been handled by the primary virtualized device further comprises, generating the statistical model for TCP network traffic and monitoring current TCP network traffic handled by the primary virtualized device. ### Claim 9 (depends on 4) The method as in claim 4, wherein monitoring the functional state of the primary virtualized device further comprises, comparing a network traffic metric representing a number of current half-open TCP connections to a data point in the statistical model corresponding to a time frame of the network traffic metric representing the number of half-open TCP connections. ### Claim 10 (depends on 4) The method as in claim 4, wherein monitoring the functional state of the primary virtualized device further comprises, comparing a network traffic metric representing a number of current TCP connections to a data point in the statistical model corresponding to a time frame of the network traffic metric representing the number of TCP connections. ### Claim 11 (depends on 4) The method as in claim 4, further comprising determining that the primary virtualized device is in the non-functional state as a result of network traffic metrics representing a number of current TCP connections and current half-open TCP connections falling below a number of TCP connections and half-open TCP connections represented by the functional state baseline. ### Claim 12 (depends on 5) The method as in claim 5, further comprising disassociating a network address from the primary virtualized device and assigning the network address to the secondary virtualized device as a result of determining that the primary virtualized device is in the non-functional state. ### Claim 13 (depends on 12) The method as in claim 12, further comprising updating a route in a software defined network route table to route network traffic to the secondary virtualized device as a result of determining that the primary virtualized device is in the non-functional state. ### Claim 14 (depends on 12) The method as in claim 12, further comprising disassociating a virtual network interface from the primary virtualized device and associating the virtual network interface with the secondary virtualized device as a result of determining that the primary virtualized device is in the non-functional state. ### Claim 15 (depends on 4) The method as in claim 4, wherein the steps of claim 4 are performed within a service provider network to monitor services that provide computing services to customers. ### Claim 16 (depends on 4) The method as in claim 4, wherein the primary virtualized device is used to manage network traffic for a virtual private network hosted by at least one remote server. ### Claim 17 A system comprising: at least one processor; a memory device including instructions that, when executed by the at least one processor, cause the system to: generate a statistical model using historical data and the at least one processor, the statistical model containing a functional state baseline representing a functional state of a primary virtualized network device and the historical data being for previous network traffic that has been handled by the primary virtualized network device included in a virtual network during a time frame; add a deviation to the functional state baseline resulting in increasing a tolerance of the functional state baseline; monitor the functional state of the primary virtualized network device using a hypervisor configured to (i) calculate network traffic metrics for current network traffic transmitted through a network layer of the hypervisor and handled by the primary virtualized network device, and (ii) compare the network traffic metrics for the current network traffic handled by the primary virtualized network device with data points in the statistical model corresponding to the network traffic metrics; determine that the primary virtualized network device is in a non-functional state; and reassign a virtual network interface from the primary virtualized network device to a secondary virtualized network device as a result of having detected that the primary virtualized network device is in the non-functional state. ### Claim 18 (depends on 17) The system as in claim 17, wherein the memory device includes instructions that, when executed by the at least one processor, causes the system to further failover to the secondary virtualized network device, the secondary virtualized network device having network state information that is synchronized with the primary virtualized network device. ### Claim 19 (depends on 17) The system as in claim 17, wherein the memory device includes instructions that, when executed by the at least one processor, causes the system to further analyze network packets transmitted to the primary virtualized network device and network packets transmitted from the primary virtualized network device to collect the historical data used to generate the statistical model.