Metadata search via n-gram index
AssigneeSnowflake Inc.
InventorsLin Chan, Tianyi Chen, Benoit Dageville, Yiming Kang, Jun Luo, Nithin Mahesh, Eric Robinson, Brian Smith
Life of this patent
Estimating the term…
Abstract
As described herein, a N-Gram index may be created and the search may be conducted using the index, which will lead to faster search results. The N-Gram index may also include partial N-Gram components to capture more relevant data. Moreover, as described herein, the search may also take into account recent log data that has not yet been indexed. Techniques for building an index store using log data and efficiently searching the index store and log data to process search requests are described herein.
Claims20
A method comprising:
- receiving, at a database system, a search request comprising a search substring;
- loading token statistics associated with N-Gram tokens in an index, the token statistics indicating a frequency of occurrence for each N-gram token;
- selecting a search strategy for processing the search request based on the token statistics;
- retrieving index records from the N-Gram index matching the search substring according to the selected search strategy;
- and
- generating search results for the search request based on the retrieved index records.
A system comprising:
- at least one hardware processor;
- and
- at least one memory storing instructions that, when executed by the at least one hardware processor, cause the at least one hardware processor to perform operations comprising:
- receiving, at a database system, a search request comprising a search substring;
- loading token statistics associated with N-Gram tokens in an index, the token statistics indicating a frequency of occurrence for each N-gram token;
- selecting a search strategy for processing the search request based on the token statistics;
- retrieving index records from the N-Gram index matching the search substring according to the selected search strategy;
- and
- generating search results for the search request based on the retrieved index records.
A machine-storage medium embodying instructions that, when executed by a machine, cause the machine to perform operations comprising:
- receiving, at a database system, a search request comprising a search substring;
- loading token statistics associated with N-Gram tokens in an index, the token statistics indicating a frequency of occurrence for each N-gram token;
- selecting a search strategy for processing the search request based on the token statistics;
- retrieving index records from the N-Gram index matching the search substring according to the selected search strategy;
- and
- generating search results for the search request based on the retrieved index records.
Dependencies are inferred from the claim text.