Indexing and querying semi-structured documents using a key-value store

A non-transitory computer readable storage medium storing instructions that when executed by one or more computer processors, cause the one or more computer processors to performs steps comprising:

• storing a plurality of indexes based on documents, each document having a document identifier and comprising fields, wherein each field has field data comprising one or more of:
• a field name, a field value, and a field type, wherein each index comprises key-value pairs stored in a key-value store, each index associated with a respective set of attributes selected from:
• (1) a document identifier attribute, (2) a field name attribute, (3) a field value attribute, and (4) a field type attribute, the set of attributes combined for obtaining keys of key-value pairs of the index, the plurality of indexes comprising:
• a document index configured to identify documents matching a criteria based on fields;
• a field index configured to access fields of a given document;
• and
• a columnar index configured to determine aggregate values associated with fields;
• receiving, from a query source, a query requesting information stored in documents indexed using the plurality of indexes, the query specifying a search criteria;
• selecting an index from the plurality of indexes for processing the query based on a type of the query;
• generating a key-expression by combining the set of attributes corresponding to the selected index;
• accessing the requested information using the selected index, the accessing comprising identifying key-value pairs matching the generated key-expression;
• and
• transmitting the accessed information to the query source.

A computer system comprising:

• one or more computer processors;
• and
• a non-transitory computer readable storage medium storing instructions that when executed by the one or more computer processors, cause the one or more computer processors to perform steps comprising:
• storing a plurality of indexes based on documents, each document having a document identifier and comprising fields, wherein each field has field data comprising one or more of:
• a field name, a field value, and a field type, wherein each index comprises key-value pairs stored in a key-value store, each index associated with a respective set of attributes selected from:
• (1) a document identifier attribute, (2) a field name attribute, (3) a field value attribute, and (4) a field type attribute, the set of attributes combined for obtaining keys of key-value pairs of the index, the plurality of indexes comprising:
• a document index configured to identify documents matching a criteria based on fields;
• a field index configured to access fields of a given document;
• and
• a columnar index configured to determine aggregate values associated with fields;
• receiving, from a query source, a query requesting information stored in documents indexed using the plurality of indexes, the query specifying a search criteria;
• selecting an index from the plurality of indexes for processing the query based on a type of the query;
• generating a key-expression by combining the set of attributes corresponding to the selected index;
• accessing the requested information using the selected index, the accessing comprising identifying key-value pairs matching the generated key-expression;
• and
• transmitting the accessed information to the query source.