System and method for data redistribution in a database
AssigneeFuturewei Technologies, Inc.
InventorsLe Cai, QingQing Zhou, Yang Sun
Life of this patent
Estimating the term…
Abstract
A method for data redistribution of a job data in a first datanode (DN) to at least one additional DN in a Massively Parallel Processing (MPP) Database (DB) is provided. The method includes recording a snapshot of the job data, creating a first data portion in the first DN and a redistribution data portion in the first DN, collecting changes to a job data copy stored in a temporary table, and initiating transfer of the redistribution data portion to the at least one additional DN.
Claims20
A method for data redistribution of job data in a source data node (DN) to at least one destination DN in a Massively Parallel Processing (MPP) Database (DB), the method comprising:
- recording a snapshot of the job data, the snapshot including job data information;
- splitting the job data of the source DN into a plurality of data portions, the data portions comprising a first data portion in the source DN and a second data portion in the source DN, and the snapshot comprising information about the split of the job data;
- collecting job data copy changes;
- identifying one or more first changes to the first data portion and identifying one or more second changes to the second data portion from the collected changes and based on the snapshot;
- initiating transfer of the second data portion from the source DN to the at least one destination DN;
- determining the data transfer of the second data portion to the at least one destination DN has failed;
- merging the identified one or more first changes into the first data portion in the source DN;
- and
- merging the identified one or more second changes into the second data portion in the source DN based on the failure of the data transfer of the second data portion to the at least one destination DN.
A device for data redistribution of job data in a source data node (DN) to at least one destination DN in a Massively Parallel Processing (MPP) Database (DB), comprising:
- a non-transitory memory storing instructions;
- and
- at least one processor in communication with the memory, the at least one processor configured, upon execution of the instructions, to perform the following steps:
- record a snapshot of the job data, the snapshot including job data information;
- split the job data of the source DN into a plurality of data portions, the data portions comprising a first data portion in the source DN and a second data portion in the source DN, and the snapshot comprising information about the split of the job data;
- collect job data copy changes;
- identify one or more first changes to the first data portion and identify one or more one or more second changes to the second data portion from the collected changes and based on the snapshot;
- initiate transfer of the second data portion from the source DN to the at least one destination DN;
- determining the data transfer of the second data portion to the at least one destination DN has failed;
- merge the identified one or more first changes into the first data portion in the source DN;
- and
- merge the identified one or more second changes into the second data portion in the source DN based on the failure of the data transfer of the second data portion to the at least one destination DN.
Dependencies are inferred from the claim text.