Back to results
US 10,884,793 B2Granted
Parallelization of data processing
AssigneeInternational Business Machines Corporation
InventorsNing Duan, Wei Huang, Peng Ji, Yi Qi, Qi Zhang, Jun Zhu
Life of this patent
Estimating the term…
Abstract
A method and apparatus for parallelization of data processing. The method including: parsing a data processing flow to split a write table sequence for the data processing flow; generating a plurality of instances of the data processing flow based at least in part on the split write table sequence; and scheduling the plurality of instances for parallelization of data processing.
Claims16
A method for parallelization of data processing, the method comprising:
- parsing a data processing flow to split a write table sequence for the data processing flow, wherein the write table sequence is split into a plurality of segments, and each neighboring segment of the plurality of segments respectively indicate different database tables;
- generating a plurality of instances of the data processing flow based on a frequency of a database table appearing in the write table sequence;
- and
- scheduling the plurality of instances for parallelization of data processing with pipeline technology, wherein the plurality of instances perform write operations on a different database tables at the same time.
1 more element
An apparatus for parallelization of data processing, the apparatus comprising:
- a memory;
- a processor device communicatively coupled to the memory;
- and
- a module configured for parallelization of data processing coupled to the memory and the processor device to carry out the steps of a method comprising:
- parsing a data processing flow to split a write table sequence for the data processing flow, wherein the write table sequence is split into a plurality of segments, and each neighboring segment of the plurality of segments respectively indicate different database tables;
- generating a plurality of instances of the data processing flow based on a frequency of a database table appearing in the write table sequence;
- and
- scheduling the plurality of instances for parallelization of data processing with pipeline technology, wherein the plurality of instances perform write operations on different database tables at the same time.
5 more elements
A computer program product comprising:
- a computer readable storage medium readable by one or more processing circuit and storing instructions for execution by one or more processor for performing a method comprising:
- parsing a data processing flow to split a write table sequence for the data processing flow, wherein the write table sequence is split into a plurality of segments, and each neighboring segment of the plurality of segments respectively indicate different database tables:
- generating a plurality of instances of the data processing flow based on a frequency of a database table appearing in the write table sequence;
- and
- scheduling the plurality of instances for parallelization of data processing with pipeline technology, wherein the plurality of instances perform write operations on different database tables at the same time.
2 more elements
Dependencies are inferred from the claim text.