Back to results
US 11,636,110 B1Granted
Metadata search via N-Gram index
AssigneeSnowflake Inc.
InventorsLin Chan, Tianyi Chen, Benoit Dageville, Yiming Kang, Jun Luo, Nithin Mahesh, Eric Robinson, Brian Smith
Life of this patent
Estimating the term…
Abstract
As described herein, a N-Gram index may be created and the search may be conducted using the index, which will lead to faster search results. The N-Gram index may also include partial N-Gram components to capture more relevant data. Moreover, as described herein, the search may also take into account recent log data that has not yet been indexed. Techniques for building an index store using log data and efficiently searching the index store and log data to process search requests are described herein.
Claims24
A method comprising:
- in response to an operation being performed on a source table, creating a log associated with the operation;
- based on the log, creating or updating an index record for a N-Gram index associated with the source table to generate a new version of the N-Gram index;
- based on the index record, updating metadata associated with the N-Gram index to facilitate searching of the N-Gram index;
- receiving, from a user, a search request including a search string, and
- in response to the search request including the search string, processing the search request by performing steps including:
- loading, in a log cache, log data associated with new logs not yet reflected in the N-Gram index,
- retrieving index records matching the search string and storing the index records in an index cache,
- merging the log data in the log cache with the index records in the index cache to generate merged data,
- performing a permission check on the merged data to determine that the user has access rights to the merged data, and
- generating results of the search request based on the merged data and the permission check,
- wherein the new logs are updated in the N-gram index in a separate process than processing the search request.
8 more elements
A system comprising:
- at least one hardware processor;
- and
- at least one memory storing instructions that, when executed by the at least one hardware processor, cause the at least one hardware processor to perform operations comprising:
- in response to an operation being performed on a source table, creating a log associated with the operation;
- based on the log, creating or updating an index record for a N-Gram index associated with the source table to generate a new version of the N-Gram index;
- based on the index record, updating metadata associated with the N-Gram index to facilitate searching of the N-Gram index;
- receiving, from a user, a search request including a search string, and
- in response to the search request including the search string, processing the search request by performing steps including:
- loading, in a log cache, log data associated with new logs not yet reflected in the N-Gram index,
- retrieving index records matching the search string and storing the index records in an index cache,
- merging the log data in the log cache with the index records in the index cache to generate merged data,
- performing a permission check on the merged data to determine that the user has access rights to the merged data, and
- generating results of the search request based on the merged data and the permission check,
- wherein the new logs are updated in the N-gram index in a separate process than processing the search request.
11 more elements
A machine-storage medium embodying instructions that, when executed by a machine, cause the machine to perform operations comprising:
- in response to an operation being performed on a source table, creating a log associated with the operation;
- based on the log, creating or updating an index record for a N-Gram index associated with the source table to generate a new version of the N-Gram index;
- based on the index record, updating metadata associated with the N-Gram index to facilitate searching of the N-Gram index;
- receiving, from a user, a search request including a search string, and
- in response to the search request including the search string, processing the search request by performing steps including:
- loading, in a log cache, log data associated with new logs not yet reflected in the N-Gram index,
- retrieving index records matching the search string and storing the index records in an index cache,
- merging the log data in the log cache with the index records in the index cache to generate merged data,
- performing a permission check on the merged data to determine that the user has access rights to the merged data, and
- generating results of the search request based on the merged data and the permission check,
- wherein the new logs are updated in the N-gram index in a separate process than processing the search request.
8 more elements
Dependencies are inferred from the claim text.