Dictionary Filtering and Evaluation in Columnar Databases

A system comprising:

• a processor;
• and
• a memory coupled with the processor and comprising stored instructions, the stored instructions executable by a processor and when executed cause the system to:
• receive a request to perform a query, the query comprising at least an operator and a request to return information about a value of interest in a columnar dataset stored on cloud storage, wherein:
• the query is associated with a filter generated to apply to the columnar dataset in response to the query,
• at least one column in the columnar dataset is based on a dictionary, and
• the dictionary maps one or more values for a column to one or more respective identifiers,
• calculate a metric based on at least a size of the dictionary and an estimated cost of evaluating the filter on the dictionary,
• compare the metric to a threshold;
• and
• execute, based on the comparison of the metric to the threshold, one of:
• performance of dictionary filtering by evaluating the filter on the dictionary before evaluating the filter on the column, and
• retrieval of the column associated with the dictionary and
• determination of whether the value of interest is present in the column when the metric is equal to or above a threshold.

A non-transitory computer readable medium comprising stored instructions encoded thereon that, when executed by a processor, cause at least one processor to:

• receive a request to perform a query, the query comprising at least an operator and a request to return information about a value of interest in a columnar dataset stored on cloud storage, wherein
• the query is associated with a filter generated to apply to the columnar dataset in response to the query,
• at least one column in the columnar dataset is based on a dictionary, and
• the dictionary maps one or more values for a column to one or more respective identifiers,
• calculate a metric based on at least a size of the dictionary and an estimated cost of evaluating the filter on the dictionary;
• compare the metric to a threshold;
• and
• execute, based on the comparison of the metric to the threshold, one of:
• performance of dictionary filtering by evaluating the filter on the dictionary before evaluating the filter on the column;
• and,
• retrieval of the column associated with the dictionary and determination of whether the value of interest is present in the column.