Methodology to improve the anomaly detection rate

A method for improving anomaly detection rate in a communication network, the method comprising:

• receiving, by a server computer, a dataset comprising traffic flows communicated over the communication network;
• grouping, by the server computer, the traffic flows into different data categories according to application layer protocols associated with the traffic flows, such that traffic flows that are grouped into a same data category are associated with a same application layer protocol, wherein each of the different data categories is associated with a set of common features that have been predetermined for traffic in an associated data category, and different data categories are associated with different sets of common features;
• detecting anomaly of the traffic flows communicated over the communication network by detecting anomaly of traffic flows that have been grouped into each data category using the set of common features associated with a corresponding category, wherein detecting the anomaly of the traffic flows communicated over the communication network comprises:
• identifying a first traffic flow in a first data category that has features that are inconsistent with the set of common features associated with the first data category;
• and
• detecting whether the first traffic flow in the first data category has anomaly based on inconsistency of the features of the first traffic flow with the set of common features associated with the first data category;
• and
• building a prediction model using the first data category and the first traffic flow for detecting anomalies in other traffic flows categorized in the first data category.

A server computer configured to improve anomaly detection rate in a communication network, the server computer comprising:

• a non-transitory memory storage comprising instructions;
• and
• one or more processors in communication with the memory storage, wherein the one or more processors execute the instructions to:
• receive a dataset comprising traffic flows communicated over the communication network;
• group the traffic flows into different data categories according to transport control protocol (TCP) port numbers or User Datagram Protocol (UDP) port numbers of the traffic flows, such that traffic flows that are grouped into a same data category have a same TCP port number or UDP port number, wherein each of the different data categories is associated with a set of common features that have been predetermined for traffic in an associated data category, and different data categories are associated with different sets of common features;
• detect anomaly of the traffic flows communicated over the communication network by detecting anomaly of traffic flows that have been grouped into each data category using the set of common features associated with a corresponding category, wherein detecting the anomaly of the traffic flows communicated over the communication network comprises:
• identifying a first traffic flow in a first data category that has features that are inconsistent with the set of common features associated with the first data category;
• and
• detecting whether the first traffic flow in the first data category has anomaly based on inconsistency of the features of the first traffic flow with the set of common features associated with the first data category;
• and
• build a prediction model using the first data category and the first traffic flow for detecting anomalies in other traffic flows categorized in the first data category.

A server computer configured to improve anomaly detection rate in a communication network, the server computer comprising:

• a non-transitory memory storage comprising instructions;
• and
• one or more processors in communication with the memory storage, wherein the one or more processors execute the instructions to:
• receive a dataset comprising traffic flows communicated over the communication network;
• group the traffic flows into different data categories according to application layer protocols associated with the traffic flows, such that traffic flows that are grouped into a same data category are associated with a same application layer protocol, wherein each of the different data categories is associated with a set of common features that have been predetermined for traffic in an associated data category, and different data categories are associated with different sets of common features;
• detect anomaly of the traffic flows communicated over the communication network by detecting anomaly of traffic flows that have been grouped into each data category using the set of common features associated with a corresponding category, wherein detecting the anomaly of the traffic flows communicated over the communication network comprises:
• identifying a first traffic flow in a first data category that has features that are inconsistent with the set of common features associated with the first data category;
• and
• detecting whether the first traffic flow in the first data category has anomaly based on inconsistency of the features of the first traffic flow with the set of common features associated with the first data category;
• and
• build a prediction model using the first data category and the first traffic flow for detecting anomalies in other traffic flows categorized in the first data category.