Optimized data structures of a relational cache with a learning capability for accelerating query execution by a data system
AssigneeDremio Corporation
InventorsTomer Shiran, Jacques Nadeau, Steven Michael Phillips
Life of this patent
Estimating the term…
Abstract
A method performed by a data system includes automatically learning relationship(s) among datasets based on one or more of a user query or an observation of a data flow through the data system. The method further includes generating an optimized data structure based on the learned relationships among the datasets. The data system then modifies a query plan to obtain query results that satisfy a query by reading the optimized data structure in lieu of reading the datasets.
Claims23
A computer-readable storage medium, excluding transitory signals and carrying instructions, which, when executed by at least one data processor of a data system, cause the data system to:
- automatically determine relationships among multiple datasets based on multiple virtual datasets,
- wherein each of the multiple virtual datasets includes a query and a reference to a dataset of the multiple datasets;
- generate an optimized data structure based on the relationships among the multiple datasets,
- wherein the optimized data structure is stored in a non-volatile memory of the data system;
- receive multiple queries;
- determine whether the optimized data structure includes one or more datasets in common with query plans for the multiple queries;
- modify each query plan that has the one or more datasets in common to read the optimized data structure stored in the non-volatile memory in lieu of reading the multiple datasets;
- and
- return query results that satisfy the multiple queries based on the modified query plans.
A non-transitory, computer-readable storage medium storing instructions, which, when executed by at least one data processor of a data system, cause the data system to:
- automatically determine relationships among multiple datasets based on queries processed by the data system or an observation of a data flow of the data system;
- generate an optimized data structure based on the relationships among the multiple datasets,
- wherein the optimized data structure is stored in a non-volatile memory of the data system;
- receive multiple queries;
- determine that the optimized data structure includes a particular dataset in common with a respective query plan of the each of the multiple queries;
- modify each respective query plan that has the particular dataset to read the optimized data structure stored in the non-volatile memory;
- and
- return query results that satisfy the multiple queries based on the modified query plans.
A method performed by a data system comprising:
- automatically determining relationships among multiple datasets based on queries processed by the data system or an observation of a data flow of the data system;
- generating an optimized data structure based on the relationships among the multiple datasets,
- wherein the optimized data structure is stored in a non-volatile memory of the data system;
- receiving multiple queries;
- determining whether the optimized data structure includes a particular dataset in common with a respective query plan of each of the multiple queries;
- modifying each respective query plan that has the particular dataset to read the optimized data structure stored in the non-volatile memory;
- and
- returning a respective query result that satisfies each of the multiple query based on a respective modified query plan.
Dependencies are inferred from the claim text.