# US 10,769,006 B2 · Ensemble risk assessment method for networked devices

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,769,006 B2
- Document ID: US10769006B2
- Status: Granted
- Kind: B2
- Application number: 16/050,370
- Filed: 2018-07-31
- Published: 2020-02-06
- Granted: 2020-09-08
- Earliest claimed: 2018-07-31
- Assignees: Cisco Technology, Inc.
- Applicants: Cisco Technology, Inc.
- Inventors: Nidhi Kao; Ulf Vinneras; John W. Garrett, Jr.
- Web page: https://patentgrep.com/patent/US10769006B2

## CPC

- G06Q 10/0635: Risk analysis of enterprise or organisation activities
- G06F 11/008: {Reliability or availability analysis}
- G06F 11/0709: {in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems}
- G06F 11/0751: {Error or fault detection not based on redundancy (power supply failures G06F1/30; network fault management H04L41/06)}
- G06F 17/18: for evaluating statistical data {, e.g. average values, frequency distributions, probability functions, regression analysis (forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04)}
- G06F 18/23213: with fixed number of clusters, e.g. K-means clustering
- G06F 18/24147: {Distances to closest patterns, e.g. nearest neighbour classification}
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/0499: Feedforward networks
- G06N 3/09: Supervised learning
- G06N 20/00: Machine learning
- G06N 20/20: Ensemble learning
- H04L 41/0816: the condition being an adaptation, e.g. in response to network events
- H04L 41/147: for predicting network behaviour
- H04L 41/149: for prediction of maintenance
- H04L 41/16: using machine learning or artificial intelligence
- G06N 3/045: Combinations of networks
- G06N 3/08: Learning methods
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- H04L 41/0853: by actively collecting configuration information or by backing up configuration information
- H04L 43/0817: by checking functioning
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

A management entity receives device fingerprints representing corresponding devices connected to one or more networks. Each device fingerprint includes a multi-bit word indicating hardware, software, network configuration, and failure features for a corresponding one of the devices. The management entity processes the device fingerprints using different methods including statistical risk of failure scoring methods and machine learning risk of failure scoring methods, to produce from each of the methods a respective risk of failure for each device. The management entity combines the respective risk of failures for each device into a composite risk of failure for each device, ranks the devices based on the composite risk of failures for the devices, to produce a risk ranking of the devices, and outputs the risk ranking.

## Claims

### Claim 1

A computer-implemented method comprising:
at a management entity:
receiving device fingerprints representing corresponding devices connected to one or more networks, each device fingerprint including a multi-bit word indicating hardware, software, network configuration, and failure features for a corresponding one of the devices;
processing the device fingerprints using statistical risk of failure scoring methods to produce risk of failures for each device;
processing the device fingerprints using machine learning risk of failure scoring methods that are different from the statistical risk of failure scoring methods to produce risk of failures for each device;
combining into a composite risk of failure for each device the risk of failures produced by the statistical risk of failure scoring methods and the risk of failures produced by the machine learning risk of failure scoring methods;
ranking the devices based on the composite risk of failures for the devices, to produce a risk ranking of the devices; and
outputting the risk ranking.

### Claim 2 (depends on 1)

The method of claim 1, wherein:
the processing using the statistical risk of failure scoring methods includes processing using one or more types of cluster profile composition scoring methods and using a nearest neighbor scoring method; and
the processing using the machine learning risk of failure scoring methods includes processing using one or more of tree-based machine learning scoring and perceptron-based machine learning scoring methods.

### Claim 3 (depends on 2)

The method of claim 2, wherein:
the one or more types of cluster profile composition scoring methods include Kmeans based on Euclidean distance, Latent Semantic Indexing analysis, and Latent Dirichlet Allocation analysis.

### Claim 4 (depends on 2)

The method of claim 2, wherein each type of cluster profile composition scoring method includes:
clustering the device fingerprints into multiple clusters based on the hardware, the software, and the network configuration features indicated by the device fingerprints;
generating for each cluster a respective statistical cluster failure rate based on the failure features of all of the device fingerprints in the cluster; and
assigning to each device fingerprint the respective statistical cluster failure rate of the cluster to which the device fingerprint belongs as the respective risk of failure of the device corresponding to the device fingerprint.

### Claim 5 (depends on 4)

The method of claim 4, wherein the clustering includes clustering the device fingerprints into multiple cluster risk groups based on the hardware, the software, and the network configuration, but not the failure features, of the device fingerprints.

### Claim 6 (depends on 2)

The method of claim 2, wherein each machine learning risk of failure scoring method includes:
training the machine learning risk of failure scoring method based on the hardware, the software, the network configuration, and the failure features of a training set of device fingerprints, to produce classification models configured to predict a risk of failure for a given device fingerprint based on the hardware, the software, and the network configuration features indicated in the given device fingerprint; and
using the classification model, predicting for each device fingerprint a respective risk of failure as the respective risk of failure of the device corresponding to the device fingerprint.

### Claim 7 (depends on 2)

The method of claim 2, wherein the nearest neighbor scoring method includes, for each device fingerprint:
determining similarity to each other device fingerprint;
determining which of the other device fingerprints are nearest neighbors based on results of the determining the similarity; and
determining a failure rate of the nearest neighbors based on the failure features of the nearest neighbors, and assigning the failure rate to the device fingerprint as the respective risk of failure of the device corresponding to the device fingerprint.

### Claim 8 (depends on 2)

The method of claim 2, wherein each device fingerprint includes the multi-bit word having bits that represent respective ones of the hardware, the software, the network configuration, and the failure features of the corresponding device, and wherein the bit of the multi-bit word that represents the failure features indicates whether or not the corresponding device has experienced a failure.

### Claim 9 (depends on 8)

The method of claim 8, wherein each device fingerprint is mapped to a device identifier of the corresponding device represented by the device fingerprint.

### Claim 10 (depends on 9)

The method of claim 9, wherein the bits of the multi-bit word of each device fingerprint have respective values that represent absences or presences of the respective ones of the hardware, the software, and the network configuration features.

### Claim 11 (depends on 1)

The method of claim 1, wherein the risk ranking ranks the devices from a high risk of failure to a low risk failure based on the composite risks of failure for the devices, and the method further comprises:
sending recommendations to reconfigure one or more features of one or more of the devices having respective risk of failures coinciding with the high risk of failure in the risk ranking; and
reconfiguring the one or more features of the one or more of the devices in accordance with recommendations to reduce risk of failures of the devices.

### Claim 12

An apparatus comprising:
a network interface unit configured to send data to and receive data from a network; and
a processor coupled to the network interface unit and configured to:
receive fingerprints representing corresponding devices connected to one or more networks, each device fingerprint including a multi-bit word indicating hardware, software, network configuration, and failure features for a corresponding one of the devices;
process the device fingerprints using statistical risk of failure scoring methods to produce risk of failures for each device;
process the device fingerprints using machine learning risk of failure scoring methods that are different from the statistical risk of failure scoring methods to produce risk of failures for each device;
combine into a composite risk of failure for each device the risk of failures produced by the statistical risk of failure scoring methods and the risk of failures produced by the machine learning risk of failure scoring methods;
rank the devices based on the composite risk of failures for the devices, to produce a risk ranking of the devices; and
output the risk ranking.

### Claim 13 (depends on 12)

The apparatus of claim 12, wherein the processor is configured to:
process using the statistical risk of failure scoring methods by processing using one or more types of cluster profile composition scoring methods and using a nearest neighbor scoring method; and
process using the machine learning risk of failure scoring methods by processing using one or more of tree-based machine learning scoring and perceptron-based machine learning scoring methods.

### Claim 14 (depends on 13)

The apparatus of claim 13, wherein:
the one or more types of cluster profile composition scoring methods include Kmeans based on Euclidean distance, Latent Semantic Indexing analysis, and Latent Dirichlet Allocation analysis.

### Claim 15 (depends on 13)

The apparatus of claim 13, wherein the processor is configured to process using each type of cluster profile composition scoring method by:
clustering the device fingerprints into multiple clusters based on the hardware, the software, and the network configuration features indicated by the device fingerprints;
generating for each cluster a respective statistical cluster failure rate based on the failure features of all of the device fingerprints in the cluster; and
assigning to each device fingerprint the respective statistical cluster failure rate of the cluster to which the device fingerprint belongs as the respective risk of failure of the device corresponding to the device fingerprint.

### Claim 16 (depends on 13)

The apparatus of claim 13, wherein the processor is configured to process using each machine learning risk of failure scoring method by:
training the machine learning risk of failure scoring method based on the hardware, the software, the network configuration, and the failure features of a training set of device fingerprints, to produce classification models configured to predict a risk of failure for a given device fingerprint based on the hardware, the software, and the network configuration features indicated in the given device fingerprint; and
using the classification model, predicting for each device fingerprint a respective risk of failure as the respective risk of failure of the device corresponding to the device fingerprint.

### Claim 17 (depends on 13)

The apparatus of claim 13, wherein the processor is configured to process using the nearest neighbor scoring method by, for each device fingerprint:
determining similarity to each other device fingerprint;
determining which of the other device fingerprints are nearest neighbors based on results of the determining the similarity; and
determining a failure rate of the nearest neighbors based on the failure features of the nearest neighbors, and assigning the failure rate to the device fingerprint as the respective risk of failure of the device corresponding to the device fingerprint.

### Claim 18 (depends on 13)

The apparatus of claim 13, wherein each device fingerprint includes the multi-bit word having bits that represent respective ones of the hardware, the software, the network configuration, and the failure features of the corresponding device, and wherein the bit of the multi-bit word that represents the failure features indicates the failure feature indicates whether or not the corresponding device has experienced a failure.

### Claim 19

A non-transitory computer readable medium encoded with instructions that, when executed by a processor, cause the processor to perform:
receiving device fingerprints representing corresponding devices connected to one or more networks, each device fingerprint including a multi-bit word indicating hardware, software, network configuration, and failure features for a corresponding one of the devices;
processing the device fingerprints using statistical risk of failure scoring methods to produce risk of failures for each device;
processing the device fingerprints using machine learning risk of failure scoring methods that are different from the statistical risk of failure scoring methods to produce risk of failures for each device;
combining into a composite risk of failure for each device the risk of failures produced by the statistical risk of failure scoring methods and the risk of failures produced by the machine learning risk of failure scoring methods;
ranking the devices based on the composite risk of failures for the devices, to produce a risk ranking of the devices; and
outputting the risk ranking.

### Claim 20 (depends on 19)

The non-transitory computer readable medium of claim 19, wherein:
the processing using the statistical risk of failure scoring methods includes processing using one or more types of cluster profile composition scoring methods and using a nearest neighbor scoring method; and
the processing using the machine learning risk of failure scoring methods includes processing using one or more of tree-based machine learning scoring and perceptron-based machine learning scoring methods.