Network security system

A method comprising:

• detecting, in a real-time processing mode, first indicia of possible security breaches based on first event data indicative of activity on a computer network, by using a computation engine implemented using Apache Storm or Apache Spark Streaming;
• generating real-time analysis result data representing the first indicia for output to a user;
• storing the real-time analysis result data in a Hadoop Distributed File System (HDFS) of an Apache Hadoop framework;
• retrieving, from the HDFS, the real-time analysis result data and second event data indicative of activity on the computer network, the second event data having been stored in the HDFS prior to analysis of the first event data by the computation engine;
• and
• detecting, in a batch mode, second indicia of possible security breaches based on the second event data and the real-time analysis result data, by using an Apache Spark cluster computing engine concurrently with use of the computation engine.

A non-transitory machine-readable storage medium storing instructions, execution of which in a computer system causes the computer system to perform operations comprising:

• detecting, in a real-time processing mode, first indicia of possible security breaches based on first event data indicative of activity on a computer network, by executing a computation engine implemented using Apache Storm or Apache Spark Streaming;
• generating real-time analysis result data representing the first indicia for output to a user;
• storing the real-time analysis result data in a Hadoop Distributed File System (HDFS) of an Apache Hadoop framework;
• retrieving, from the non-volatile storage system, the real-time analysis result data and second event data indicative of activity on the computer network, the second event data having been stored in the HDFS prior to analysis of the first event data by the computation engine;
• and
• detecting, in a batch mode, second indicia of possible security breaches based on the second event data and the real-time analysis result data, by executing an Apache Spark cluster computing engine concurrently with the executing of the computation engine.