Multidimension column-based partitioning and storage

A security information and event management system comprising:

• a storage engine executed by at least one processor to partition event data across multiple dimensions simultaneously, to determine chunks according to the partitioning, and to perform column-based storage of the chunks, wherein each chunk represents portioned data in one column of a an event schema and wherein the storage engine is to store metadata for the chunks, and the metadata identifies all chunks in a stripe for each dimension, and the stripe comprises chunks for each column in the event schema;
• a query manager to receive a query, to identify stored chunks relevant. to the query according to the metadata;
• and execute the query on the identified chunks;
• and
• an event processing engine to correlate some of the column-based stored event data in accordance with rules, instructions or requests to identify security threats.

A non-volatile computer readable medium including machine readable instructions executable by at least one processor to:

• determine dimensions to partition data across multiple dimensions simultaneously;
• determine chunks for each dimension;
• perform column-based storage of the chunks, wherein each chunk represents portioned data in one column of a schema for the data;
• determine stripes for each partition, wherein each stripe comprises chunks for each column in the schema;
• and
• store metadata identifying the stripes.