# US 11,115,263 B2 · Intra-cluster node troubleshooting method and device > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,115,263 B2 - Document ID: US11115263B2 - Status: Granted - Kind: B2 - Application number: 16/732,749 - Filed: 2020-01-02 - Published: 2020-05-07 - Granted: 2021-09-07 - Earliest claimed: 2017-07-12 - Assignees: Huawei Technologies Co., Ltd. - Applicants: Huawei Technologies Co., Ltd. - Inventors: Yan Zeng; Fan Yu; Yinwen Wang; Yutao Shuai; Xiaoming Yue - Web page: https://patentgrep.com/patent/US11115263B2 ## CPC - H04L 41/065: {involving logical or physical relationship, e.g. grouping and hierarchies} - G06F 11/1425: {by reconfiguration of node membership} - H04L 41/0668: by dynamic selection of recovery network elements, e.g. replacement by the most appropriate element after failure - H04L 41/0677: Localisation of faults - H04L 41/0686: Additional information in the notification, e.g. enhancement of specific meta-data - H04L 41/12: Discovery or management of network topologies - H04L 41/40: using virtualisation of network functions or resources, e.g. SDN or NFV entities - H04L 43/0817: by checking functioning - H04L 43/0823: Errors, e.g. transmission errors - H04L 43/0829: Packet loss - H04L 43/10: Active monitoring, e.g. heartbeat, ping or trace-route - H04L 43/20: the monitoring system or the monitored elements being virtualised, abstracted or software-defined entities, e.g. SDN or NFV - G06F 11/0709: {in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems} - G06F 11/0754: {by exceeding limits} - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Embodiments of this application relate to an intra-cluster node troubleshooting method and device. The method includes: obtaining fault detection topology information of a cluster, where the fault detection topology information includes a fault detection relationship between all nodes in the cluster; obtaining a fault indication message, where the fault indication message is used to indicate unreachability from a detection node to a detected node; determining a sub-cluster of the cluster based on the fault detection topology information and the fault indication message, where nodes that belong to different sub-clusters are unreachable to each other; and determining a working cluster based on the sub-cluster of the cluster. According to the embodiments of this application, available nodes in the cluster can be retained to a maximum extent at relatively low costs. In this way, a quantity of available nodes in the cluster is increased, high availability is ensured. ## Claims ### Claim 1 A troubleshooting method for nodes in a cluster, the method comprising: obtaining fault detection topology information of the cluster, fault detection being performed on one node in the cluster by at least one other node in the cluster, the fault detection topology information comprising a fault detection relationship between a detection node and a detected node in the cluster; receiving a fault indication message from the detection node, the fault indication message indicating unreachability from the detection node to the detected node; determining a sub-cluster of the cluster based on the fault detection topology information and the fault indication message, with nodes belonging to different sub-clusters of the cluster being unreachable to each other; and determining a working cluster based on the determined sub-cluster. ### Claim 2 (depends on 1) The method according to claim 1, wherein the determining the working cluster based on the determined sub-cluster comprises any one of: determining, as the working cluster, a sub-cluster having a largest quantity of nodes; determining, as the working cluster, a sub-cluster comprising a seed node and having a largest quantity of nodes, wherein the seed node is a preconfigured node and a non-seed node joins the cluster using the seed node; determining, as the working cluster, a sub-cluster comprising a largest quantity of seed nodes; or determining, as the working cluster, a sub-cluster having a largest quantity of nodes running a main service. ### Claim 3 (depends on 1) The method according to claim 1, wherein the determining the working cluster based on the determined sub-cluster comprises: determining the working cluster based on a health status or a resource availability status of a node in the determined sub-cluster, the health status of the node being determined based on a period of time in which the node makes a response to a detection packet. ### Claim 4 (depends on 1) The method according to claim 1, wherein the determining the sub-cluster based on the fault detection topology information and the fault indication message comprises: determining a fault detection relationship topology between nodes based on the fault detection topology information; deleting, from the fault detection relationship topology, an edge corresponding to the fault indication message to obtain an updated fault detection relationship topology; determining a connected subgraph of the updated fault detection relationship topology; and determining the sub-cluster based on the determined connected subgraph of the updated fault detection relationship topology. ### Claim 5 (depends on 1) The method according to claim 1, wherein the determining the sub-cluster based on the fault detection topology information and the fault indication message comprises: determining a faulty node and a faulty link in the cluster based on the fault detection topology information and the fault indication message; deleting the faulty node, the faulty link, or the faulty node and the faulty link from a network topology of the cluster to obtain an updated network topology; determining a connected subgraph of the updated network topology, the updated network topology comprising information about network connections between all nodes in the cluster; and determining the sub-cluster based on the determined connected subgraph. ### Claim 6 (depends on 1) The method according to claim 1, wherein the working cluster includes a set of unreachable nodes each being a detected node that has one or more fault indication messages pointing to it, each of the one or more fault indication messages being from a detection node and indicating unreachability from the detection node to the detected node, and the method further comprises: determining, among the set of unreachable nodes in the working cluster, a first unreachable node that has a largest quantity of fault indication messages pointing to it as a to-be-deleted node; and sending a first indication message to another node in the working cluster, the first indication message indicating the to-be-deleted node. ### Claim 7 (depends on 6) The method according to claim 6, wherein the determining the first unreachable node as the to-be-deleted node comprises: determining, among the set of unreachable nodes in the working cluster, an unreachable node that has the largest quantity of the fault indication messages pointing to it and whose health status is worst, as the to-be-deleted node. ### Claim 8 (depends on 1) The method according to claim 1, wherein the obtaining the fault detection topology information of the cluster comprises: receiving the fault detection relationship sent by another node in the cluster, and determining the fault detection topology information based on the received fault detection relationship; or deducing the fault detection topology information according to a preset rule. ### Claim 9 A troubleshooting device, comprising: a transceiver configured to communicate with a detection node; a memory storing instructions; and a processor in communication with the transceiver and the memory, the processor executing the instructions to perform: obtaining fault detection topology information of a cluster, fault detection being performed on one node in the cluster by at least one other node in the cluster, the fault detection topology information comprising a fault detection relationship between the detection node and a detected node in the cluster; receiving a fault indication message from the detection node, the fault indication message indicating unreachability from the detection node to the detected node; determining a sub-cluster of the cluster based on the fault detection topology information and the fault indication message, with nodes belonging to different sub-clusters of the cluster being unreachable to each other; and determining a working cluster based on the determined sub-cluster. ### Claim 10 (depends on 9) The troubleshooting device according to claim 9, wherein the determining the working cluster based on the determined sub-cluster comprises any one of: determining, as the working cluster, a sub-cluster having a largest quantity of nodes; determining, as the working cluster, a sub-cluster comprising a seed node and having a largest quantity of nodes, wherein the seed node is a preconfigured node and a non-seed node joins the cluster using the seed node; determining, as the working cluster, a sub-cluster comprising a largest quantity of seed nodes; or determining, as the working cluster, a sub-cluster having a largest quantity of nodes running a main service. ### Claim 11 (depends on 9) The troubleshooting device according to claim 9, wherein the determining the working cluster based on the determined sub-cluster comprises: determining the working cluster based on a health status or a resource availability status of a node in the sub-cluster, the health status of the node being determined based on a period of time in which the node makes a response to a detection packet. ### Claim 12 (depends on 9) The troubleshooting device according to claim 9, wherein the determining the sub-cluster based on the fault detection topology information and the fault indication message comprises: determining a fault detection relationship topology between nodes based on the fault detection topology information; deleting, from the fault detection relationship topology, an edge corresponding to the fault indication message to obtain an updated fault detection relationship topology; determining a connected subgraph of the updated fault detection relationship topology; and determining the sub-cluster based on the determined connected subgraph of the updated fault detection relationship topology. ### Claim 13 (depends on 9) The troubleshooting device according to claim 9, wherein the determining the sub-cluster based on the fault detection topology information and the fault indication message comprises: determining a faulty node and a faulty link in the cluster based on the fault detection topology information and the fault indication message; deleting the faulty node, the faulty link, or the faulty node and the faulty link from a network topology of the cluster to obtain an updated network topology of the cluster; determining a connected subgraph of the updated network topology, the updated network topology comprising information about network connections between all nodes in the cluster; and determining the sub-cluster based on the determined connected subgraph. ### Claim 14 (depends on 9) The troubleshooting device according to claim 9, wherein the working cluster includes a set of unreachable nodes each being a detected node that has one or more fault indication messages pointing to it, each of the one or more fault indication messages being from a detection node and indicating unreachability from the detection node to the detected node, and the processor further executes the instructions to perform: determining, among the set of unreachable nodes in the working cluster, a first unreachable node that has a largest quantity of fault indication messages pointing to it as a to-be-deleted node; and sending a first indication message to another node in the working cluster, the first indication message indicating the to-be-deleted node. ### Claim 15 (depends on 14) The troubleshooting device according to claim 14, wherein the determining the first unreachable node as the to-be-deleted node comprises: determining, among the set of unreachable nodes in the working cluster, an unreachable node that has the largest quantity of the fault indication messages pointing to it and whose health status is worst, as the to-be-deleted node. ### Claim 16 (depends on 9) The troubleshooting device according to claim 9, wherein the obtaining the fault detection topology information of the cluster comprises: receiving the fault detection relationship sent by another node in the cluster, and determining the fault detection topology information based on the received fault detection relationship; or deducing the fault detection topology information according to a preset rule. ### Claim 17 A non-transitory computer readable medium storing computer-readable instructions that when executed by a processor, cause the processor to perform operations comprising: obtaining fault detection topology information of a cluster, fault detection being performed on one node in the cluster by at least one other node in the cluster, the fault detection topology information comprising a fault detection relationship between a detection node and a detected node in the cluster; receiving a fault indication message from the detection node, the fault indication message indicating unreachability from the detection node to the detected node; determining a sub-cluster of the cluster based on the fault detection topology information and the fault indication message, with nodes belonging to different sub-clusters of the cluster being unreachable to each other; and determining a working cluster based on the determined sub-cluster.