Incremental clustering of database tables
AssigneeSnowflake Inc.
InventorsJiaqi Yan, Thierry Cruanes, Jeffrey Rosen, William Waddington, Prasanna Rajaperumal, Abdul Munir
Life of this patent
Estimating the term…
Abstract
Automatic clustering of a database table is disclosed. A method for automatic clustering of a database table includes receiving an indication that a data modification task has been executed on a table and determining whether the table is sufficiently clustered. The method includes, in response to determining the table is not sufficiently clustered, selecting one or more micro-partitions of the table to be reclustered. The method includes assigning each of the one or more micro-partitions to an execution node to be reclustered.
Claims27
A method comprising:
- receiving an indication that a data modification task has been executed on a table, the data modification task comprising ingesting new micro-partitions into the table;
- determining, subsequent to the execution of the data modification task on the table, that the table is not sufficiently clustered, the determining comprising:
- retrieving level information for the table;
- identifying, based on the retrieved level information, a proportion of micro-partitions in lower levels of the table;
- and
- determining that the identified proportion of micro-partitions in lower levels of the table exceeds a threshold, and responsively entering a catch-up mode in which reclustering operations are performed;
- selecting one or more micro-partitions of the table to be reclustered;
- and
- reclustering the one or more selected micro-partitions.
A system comprising:
- at least one processor;
- and
- one or more non-transitory computer readable storage media containing instructions executable by the at least one processor for causing the at least one processor to perform operations comprising:
- receiving an indication that a data modification task has been executed on a table, the data modification task comprises ingesting new micro-partitions into the table;
- determining, subsequent to the execution of the data modification task on the table, that the table is not sufficiently clustered, the determining comprising:
- retrieving level information for the table;
- identifying, based on the retrieved level information, a proportion of micro-partitions in lower levels of the table;
- and
- determining that the identified proportion of micro-partitions in lower levels of the table exceeds a threshold, and responsively entering a catch-up mode in which reclustering operations are performed;
- selecting one or more micro-partitions of the table to be reclustered;
- and
- reclustering the one or more selected micro-partitions.
One or more non-transitory computer readable storage media containing instructions executable by at least one processor for causing the at least one processor to perform operations comprising:
- receiving an indication that a data modification task has been executed on a table, the data modification task comprises ingesting new micro-partitions into the table;
- determining, subsequent to the execution of the data modification task on the table, that the table is not sufficiently clustered, the determining comprising:
- retrieving level information for the table;
- identifying, based on the retrieved level information, a proportion of micro-partitions in lower levels of the table;
- and
- determining that the identified proportion of micro-partitions in lower levels of the table exceeds a threshold, and responsively entering a catch-up mode in which reclustering operations are performed;
- selecting one or more micro-partitions of the table to be reclustered;
- and
- reclustering the one or more selected micro-partitions.
Dependencies are inferred from the claim text.