Distributed database with mappings between append-only files and repartitioned files
AssigneeCitus Data Bilgi Islemieri Ticaret A.S.
InventorsOzgun Ali Erdogan, Sumedh Suhas Pathak
Life of this patent
Estimating the term…
Abstract
A method implemented by a computer network includes storing a database table in a distributed database resident on the computer network. The database table is represented with append-only files, where each append-only file stores data in a logical representation that is independent of the physical representation of the data at a node of the computer network. Data in the append-only files are repartitioned across nodes in the computer network to form repartitioned files. Information about a mapping between the append-only files and the repartitioned files is maintained.
Claims17
A method implemented by a computer network, comprising;
- storing a database table in a distributed database resident on a computer network with a plurality of nodes, wherein the database table is partitioned to database table partitions across the plurality of nodes within the computer network and wherein each database table partition does not exceed a maximum size and therefore each of the plurality of nodes holds multiple database table partitions, and wherein each database table partition is ordered on a first dimension;
- representing the database table with a plurality of append-only files, wherein each append-only file stores data in a logical representation that is independent of the physical representation of the data at a node of the computer network;
- repartitioning data in the append-only files across the plurality of nodes in the computer network to form repartitioned files, wherein repartitioning includes repartitioning database table partitions based upon a second dimension to form repartition table segments that are distributed to different nodes, and merging repartition table segments at the different nodes based upon the second dimension;
- keeping information about mappings between the append-only files and the repartitioned files;
- utilizing data in the repartitioned files to construct a repartitioned database table;
- receiving a database query;
- and
- partitioning the database query to run in parallel across the repartitioned database table.
A method implemented by a computer network, comprising;
- storing a database table in a distributed database resident on a computer network with a plurality of nodes, wherein the database table is partitioned to database table partitions across the plurality of nodes within the computer network and wherein each database table partition does not exceed a maximum size and therefore each of the plurality of nodes holds multiple database table partitions, and wherein each database table partition is ordered on a first dimension;
- representing the database table with a plurality of append-only files, wherein each append-only file stores data in a logical representation that is independent of the physical representation of the data at a node of the computer network;
- repartitioning data in the append-only files across the plurality of nodes in the computer network to form repartitioned files, wherein repartitioning includes repartitioning database table partitions based upon a second dimension to form repartition table segments that are distributed to different nodes, and merging repartition table segments at the different nodes based upon the second dimension;
- keeping information about mappings between the append-only files and the repartitioned files;
- keeping statistics on data in the repartitioned files;
- receiving a database query;
- and
- using the statistics to prune a set of repartitioned files not relevant to the database query.
Dependencies are inferred from the claim text.