Retrieving event records from a field searchable data store using references values in inverted indexes

A network device that is configured to searching data, the network device comprising:

• a memory that is configured to store at least one instruction;
• and
• a processor device that is configured to execute instructions that enable actions, the actions comprising:
• evaluating a query associated with a first set of events in a field searchable data store comprising a plurality of events, each event comprising a time-stamped portion of raw machine data;
• responsive to a determination that the query is a pipelined query comprising a plurality of pipe-stages that includes a first pipe-stage and a second pipe-stage, generating results for the query, wherein the generating results comprises:
• responsive to the first pipe-stage of the query, accessing an inverted index that comprises a plurality of entries including reference values for identifying respective locations in the field searchable data store for the first set of events, and wherein a respective inverted index entry comprises:
• at least one field operable to be extracted from each of the first set of events;
• an at least one associated field value;
• and
• a reference value associated with a location in the field searchable data store of a respective event from the first set of events;
• responsive to the second pipe-stage of the query, performing a search to filter out a second set of events from the first set of events;
• and
• retrieving the second set of events from the field searchable data store using the reference values in the inverted index, wherein evaluating the query comprises:
• partitioning the pipelined query into blocks corresponding to commands;
• extracting semantic information from the commands in the pipelined query;
• constructing an abstract semantic tree with the semantic information;
• and
• performing optimization operations for the pipelined query using the semantic information, wherein the optimization operations comprise a predicate pushdown.

A processor readable non-transitive storage media that includes instructions wherein execution of the instructions by a processor device enables actions, wherein the actions comprise:

• evaluating a query associated with a first set of events in a field searchable data store comprising a plurality of events, each event comprising a time-stamped portion of raw machine data;
• responsive to a determination that the query is a pipelined query comprising a plurality of pipe-stages that includes a first pipe-stage and a second pipe-stage, generating results for the query, wherein the generating results comprises:
• responsive to the first pipe-stage of the query, accessing an inverted index that comprises a plurality of entries including reference values for identifying respective locations in the field searchable data store for the first set of events, and wherein a respective inverted index entry comprises:
• at least one field operable to be extracted from each of the first set of events;
• an at least one associated field value;
• and
• a reference value associated with a location in the field searchable data store of a respective event from the first set of events;
• responsive to the second pipe-stage of the query, performing a search to filter out a second set of events from the first set of events;
• and
• retrieving the second set of events from the field searchable data store using the reference values in the inverted index, wherein evaluating the query comprises:
• partitioning the pipelined query into blocks corresponding to commands;
• extracting semantic information from the commands in the pipelined query;
• constructing an abstract semantic tree with the semantic information;
• and
• performing optimization operations for the pipelined query using the semantic information, wherein the optimization operations comprise a predicate pushdown.