Intrusion detection using robust singular value decomposition

A system comprising:

• a memory;
• a network interface;
• and
• one or more processors in communication with the memory and the network interface, the one or more processors configured to:
• create an anomaly detection model including a singular value matrix and a data pattern matrix from a matrix of historical network traffic data;
• store the singular value matrix and the data pattern matrix of the anomaly detection model;
• receive streaming network traffic data;
• perform a log transform on the streaming network traffic data;
• apply the anomaly detection model to a matrix of the streaming network traffic data in real time as the streaming network traffic data is received;
• detect anomalous patterns in the streaming network traffic data based on patterns identified by the anomaly detection model, wherein the one or more processors is configured to detect the anomalous patterns in the streaming network traffic data by being configured to:
• perform an error calculation between a row of the matrix of the streaming network traffic data and a corresponding row of a reconstructed matrix of the streaming network traffic data;
• assign a value to a result of the error calculation based on a cumulative distribution function for a plurality of error calculation results;
• determine whether a cumulative distribution function value for the error calculation exceeds a specified threshold cumulative distribution function value;
• and
• in response to determine that the cumulative distribution function value for the error calculation exceeds the specified threshold cumulative distribution function value, identify the corresponding streaming network traffic data as anomalous;
• and
• associate the anomalous patterns in the streaming network traffic data with internet protocol addresses.

A non-transitory computer-readable storage medium storing instructions which, when executed by a processor, cause the processor to perform operations, the operations comprising:

• creating an anomaly detection model including a singular value matrix and a data pattern matrix from a matrix of historical network traffic data;
• storing the singular value matrix and the data pattern matrix of the anomaly detection model;
• receiving streaming network traffic data;
• performing a log transform on the streaming network traffic data;
• applying the anomaly detection model to a matrix of the streaming network traffic data in real time as the streaming network traffic data is received;
• detecting anomalous patterns in the streaming network traffic data based on patterns identified by the anomaly detection model, wherein the detecting the anomalous patterns in the streaming network traffic data comprises:
• performing an error calculation between a row of the matrix of the streaming network traffic data and a corresponding row of a reconstructed matrix of the streaming network traffic data;
• assigning a value to a result of the error calculation based on a cumulative distribution function for a plurality of error calculation results;
• determining whether a cumulative distribution function value for the error calculation exceeds a specified threshold cumulative distribution function value;
• and
• in response to determining that the cumulative distribution function value for the error calculation exceeds the specified threshold cumulative distribution function value, identifying the corresponding streaming network traffic data as anomalous;
• and
• associating the anomalous patterns in the streaming network traffic data with internet protocol addresses.