Using cluster processing to identify sets of similarly failing hosts
AssigneeAmazon Technologies, Inc.
InventorsJames Michael Thompson, Marc Stephen Olson, Vaibhav Sharma
Life of this patent
Estimating the term…
Abstract
Data, attributes, and metrics from unavailable resource hosts may be collected and used for cluster analysis in order to correlate the different hosts and group similar hosts into clusters. The clusters may be ranked based on the collected information and used to provide a simple way to identify shared failure modes among the unavailable hosts. By identifying the hosts of each cluster, shared failures can be corrected for large groups of hosts at the same time, enabling the hosts to return to operational states.
Claims21
A computer-implemented method, comprising:
- obtaining information identifying a plurality of resource hosts in abnormal operation;
- obtaining a data set for the plurality of resource hosts in abnormal operation, the data set comprising, for each resource host of the plurality of resource hosts in abnormal operation, values for a set of attributes associated with the resource host;
- performing a clustering algorithm, using the obtained data set, to identify a set of clusters of resource hosts;
- generating a ranking of clusters in the identified set of clusters of resource hosts, the ranking generated based on an estimate of an amount of computer processing to be performed on each cluster of the set of clusters;
- selecting a cluster based at least in part on the generated ranking;
- and
- providing information identifying resource hosts in the selected cluster.
A system, comprising at least one computing device configured to implement one or more services, wherein the one or more services are configured to:
- obtain a data set for a plurality of resource hosts identified as being in abnormal operation, the data set, for each resource host of the plurality of resource hosts, comprising at least one value for at least one attribute of the resource host;
- perform a clustering algorithm, using the data set, to group at least a subset of the plurality of resource hosts among a set of clusters;
- rank each cluster of the set of clusters of the plurality of resource hosts based at least in part on a type of storage volume associated with the each cluster;
- select a cluster based at least in part on an associated rank of the cluster;
- and
- provide information identifying resource hosts in the selected cluster.
A non-transitory computer-readable storage medium storing thereon executable instructions that, as a result of being executed by one or more processors of a computer system, cause the computer system to at least:
- obtain, for a plurality of nodes identified as being in abnormal operation, attribute types and values of the attribute types;
- use the values of the attribute types as input to a clustering algorithm to identify a set of clusters of nodes from the plurality of nodes, wherein at least one cluster of the set of clusters of nodes is associated with a possible cause of abnormal operation;
- generate a ranking of clusters of the set of clusters of nodes based at least in part on a number of nodes associated with each cluster of nodes and a customer entity associated with the each cluster of nodes;
- select a cluster based at least in part on an associated rank of the cluster;
- and
- provide information identifying nodes of the selected cluster.
Dependencies are inferred from the claim text.