# US 2020/0287919 A1 · Verifying accuracy of ml pipelines using third party co-ordination

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0287919 A1
- Document ID: US20200287919A1
- Status: Published application
- Kind: A1
- Application number: 16/293,937
- Filed: 2019-03-06
- Published: 2020-09-10
- Earliest claimed: 2019-03-06
- Assignees: Cisco Technology, Inc.
- Applicants: Cisco Technology, Inc.
- Inventors: Ram Mohan Ravindranath; Prashanth Patil; Rajesh Indira Viswambharan
- Web page: https://patentgrep.com/patent/US20200287919A1

## CPC

- H04L 63/1425: {Traffic logging, e.g. anomaly detection}
- H04L 67/10: in which an application is distributed across nodes in the network (software deployment G06F8/60; multiprogramming arrangements G06F9/46)
- G06F 16/285: {Clustering or classification}
- G06N 20/00: Machine learning
- H04L 41/16: using machine learning or artificial intelligence
- H04L 43/026: using flow identification
- H04L 45/74: Address processing for routing
- H04L 63/123: {received data contents, e.g. message integrity}
- H04L 69/22: Parsing or analysis of headers
- H04L 63/0421: {Anonymous communication, i.e. the party's identifiers are hidden from the other party or parties, e.g. using an anonymizer}
- H04L 63/145: {the attack involving the propagation of malware through the network, e.g. viruses, trojans or worms}
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Systems, methods, computer-readable media, and devices are disclosed for verifying traffic classification. At a first node, a classification to a received packet is designated according to a local model. The classification of the packet by the first node is verified by sending packet information describing the packet to a distributed network comprising multiple nodes, where the packet information includes attributes of the packet. The classification of the packet is verified from receiving results from a second node that, based on the attributes, independently classifies the packet. Based on the verified classification, decentralized information for classifying packets is updated.

## Claims

### Claim 1

A method for verifying traffic classification comprising:
designating, at a first node, a classification to a received packet according to a local model; and
verifying the classification of the packet by the first node by:
sending packet information describing the packet to a distributed network comprising multiple nodes, wherein the packet information comprises attributes of the packet;
verifying the classification of the packet from receiving results from a second node that, based on the attributes, independently classifies the packet; and
updating decentralized information for classifying packets based on the verified traffic classification.

### Claim 2 (depends on 1)

The method of claim 1, wherein verifying the classification of the packet further comprises:
receiving results from a subset of nodes of the distributed network classifying the packet, the classification according to models specific to each node;
cross-verifying the classification among the subset of nodes by matching the results; and
based on a same result from a threshold number of nodes of the subset of the nodes, confirming the classification of the packet.

### Claim 3 (depends on 1)

The method of claim 1, further comprising appending the updated decentralized information to a ledger on the distributed network, wherein the updated decentralized information comprises the verified traffic classification and the attributes of the packet.

### Claim 4 (depends on 1)

The method of claim 1, wherein the attributes of the packet are associated with a threat pattern extracted from a portion of traffic, the threat pattern extracted based on statistically based patterns describing malware.

### Claim 5 (depends on 1)

The method of claim 1, wherein the classification is verified by at least one validating node, the at least one validating node having a reputation score above a threshold value.

### Claim 6 (depends on 1)

The method of claim 1, wherein a node within the distributed network having a reputation score below a threshold value cannot verify the classification, and wherein the node having the reputation score below the threshold is restricted from modifying the decentralized information.

### Claim 7 (depends on 1)

The method of claim 1, wherein receiving results from a second node comprises receiving an independent classification of the packet, using the attributes, based on a second model providing a unique solution from the local model.

### Claim 8

A system for verifying traffic classification comprising:
a server to verify traffic classification within a network, the server to:
designate, at a first node, a classification to a received packet according to a local model; and
verify the classification of the packet by the first node by:
sending packet information describing the packet to a distributed network comprising multiple nodes, wherein the packet information comprises attributes of the packet;
verifying the classification of the packet from receiving results from a second node that, based on the attributes, independently classifies the packet; and
updating decentralized information for classifying packets based on the verified traffic classification.

### Claim 9 (depends on 8)

The system of claim 8, wherein verifying the classification of the packet further comprises the server to:
receive results from a subset of nodes of the distributed network classifying the packet, the classification according to models specific to each node;
cross-verify the classification among the subset of nodes by matching the results; and
based on a same result from a threshold number of nodes of the subset of the nodes, confirm the classification of the packet.

### Claim 10 (depends on 8)

The system of claim 8, the server further to:
append the updated decentralized information to a ledger on the distributed network, wherein the updated decentralized information comprises the verified traffic classification and the attributes of the packet.

### Claim 11 (depends on 8)

The system of claim 8, wherein the attributes of the packet are associated with a threat pattern extracted from a portion of traffic, the threat pattern extracted based on statistically based patterns describing malware.

### Claim 12 (depends on 8)

The system of claim 8, wherein the classification is verified by at least one validating node, the at least one validating node having a reputation score above a threshold value.

### Claim 13 (depends on 8)

The system of claim 8, wherein a node within the distributed network having a reputation score below a threshold value cannot verify the classification, and wherein the node having the reputation score below the threshold is restricted from modifying the decentralized information.

### Claim 14 (depends on 8)

The system of claim 8, wherein receiving results from a second node comprises receiving an independent classification of the packet, using the attributes, based on a second model providing a unique solution from the local model.

### Claim 15

A non-transitory computer-readable medium comprising instructions stored thereon, the instructions for verifying traffic classification and executable by one or more processors of a computing system to cause the computing system to:
designate, at a first node, a classification to a received packet according to a local model; and
verify the classification of the packet by the first node by:
sending packet information describing the packet to a distributed network comprising multiple nodes, wherein the packet information comprises attributes of the packet;
verifying the classification of the packet from receiving results from a second node that, based on the attributes, independently classifies the packet; and
updating decentralized information for classifying packets based on the verified traffic classification.

### Claim 16 (depends on 15)

The non-transitory computer-readable medium of claim 15, wherein verifying the classification of the packet further comprises instructions further causing the computing system to:
receive results from a subset of nodes of the distributed network classifying the packet, the classification according to models specific to each node;
cross-verify the classification among the subset of nodes by matching the results; and
based on a same result from a threshold number of nodes of the subset of the nodes, confirm the classification of the packet.

### Claim 17 (depends on 15)

The non-transitory computer-readable medium of claim 15, the instructions further causing the computing system to:
append the updated decentralized information to a ledger on the distributed network, wherein the updated decentralized information comprises the verified traffic classification and the attributes of the packet.

### Claim 18 (depends on 15)

The non-transitory computer-readable medium of claim 15, wherein the attributes of the packet are associated with a threat pattern extracted from a portion of traffic, the threat pattern extracted based on statistically based patterns describing malware.

### Claim 19 (depends on 15)

The non-transitory computer-readable medium of claim 15, wherein the classification is verified by at least one validating node, the at least one validating node having a reputation score above a threshold value.

### Claim 20 (depends on 15)

The non-transitory computer-readable medium of claim 15, wherein a node within the distributed network having a reputation score below a threshold value cannot verify the classification, and wherein the node having the reputation score below the threshold is restricted from modifying the decentralized information.
