Leveraging references values in inverted indexes to retrieve associated event records comprising raw machine data

A network device that is operative for searching data, the network device comprising:

• a memory that is operative to store at least one instruction;
• and
• a processor device that is operative to execute instructions that enable actions, the actions comprising:
• evaluating a query associated with a first set of events in a field searchable data store comprising a plurality of events, each event comprising a time-stamped portion of raw machine data;
• responsive to a determination that the query is a pipelined query comprising a plurality of pipe-stages that includes a first pipe-stage and a second pipe-stage, generating results for the query, wherein the generating results comprises:
• responsive to the first pipe-stage of the query, accessing an inverted index that comprises a plurality of entries including reference values for identifying respective locations in the field searchable data store for the first set of events, and wherein a respective inverted index entry comprises:
• at least one field operable to be extracted from each of the first set of events;
• an at least one associated field value;
• and
• a reference value associated with a location in the field searchable data store of a respective event from the first set of events;
• responsive to the second pipe-stage of the query, performing a search to filter out a second set of events from the first set of events;
• and
• retrieving the second set of events from the field searchable data
• store using the reference values in the inverted index, wherein each of the reference values identifies a location in the field searchable data store where an associated event from the second set of events is stored,
• wherein the performing the search further comprises:
• responsive to a determination that the second pipe-stage of the query comprises a predicate referencing the at least one field where the at least one field is summarized in the summarization table, evaluating the predicate against the at least one field and the at least one associated field value in each entry within the inverted index;
• and
• in accordance with the evaluating of the predicate, determining the second set of events to retrieve from the field searchable data store.

A processor readable non-transitive storage media that includes instructions wherein execution of the instructions by a processor device enables actions, wherein the actions comprise:

• evaluating a query associated with a first set of events in a field searchable data store comprising a plurality of events, each event comprising a time-stamped portion of raw machine data;
• responsive to a determination that the query is a pipelined query comprising a plurality of pipe-stages that includes a first pipe-stage and a second pipe-stage, generating results for the query, wherein the generating results comprises:
• responsive to the first pipe-stage of the query, accessing an inverted index that comprises a plurality of entries including reference values for identifying respective locations in the field searchable data store for the first set of events, and wherein a respective inverted index entry comprises:
• at least one field operable to be extracted from each of the first set of events;
• an at least one associated field value;
• and
• a reference value associated with a location in the field searchable data store of a respective event from the first set of events;
• responsive to the second pipe-stage of the query, performing a search to filter out a second set of events from the first set of events;
• and
• retrieving the second set of events from the field searchable data
• store using the reference values in the inverted index, wherein each of the reference values identifies a location in the field searchable data store where an associated event from the second set of events is stored,
• wherein the performing the search further comprises:
• responsive to a determination that the second pipe-stage of the query comprises a predicate referencing the at least one field where the at least one field is summarized in the summarization table, evaluating the predicate against the at least one field and the at least one associated field value in each entry within the inverted index;
• and
• in accordance with the evaluating of the predicate, determining the second set of events to retrieve from the field searchable data store.