System for integrating task and data parallelism in dynamic applications
AssigneeCompaq Information Technologies Group, L.P.
InventorsJames Mathew Rehg, Kathleen Knobe, Rishiyur S. Nikhil, Umakishore Ramachandran
Life of this patent
Estimating the term…
Abstract
A system for integrating task and data parallelism in a dynamic application that includes at least one task for processing an input data stream to produce an output data stream replaces the at least one task with the following components. A splitter task for partitioning the input data stream into a plurality of data chunks. A plurality of worker tasks for processing subsets of the data chunks, each worker task being an instance of the at least one task, and a joiner task combining the processed data chunks to produce the output data stream.
Claims14
A system for integrating task and data parallelism for a dynamic application including at least one task for real-time processing of a continuous input data stream to produce an output data stream, comprising:
- a splitter task partitioning the input data stream into a plurality of individually marked data chunks;
- a work queue storing the data chunks;
- a variable number of worker tasks processing subsets of the data chunks from the work queue, each worker task being an instance of the at least one task;
- a controller queue storing a control item for each data chunk, the control item including task and data strategy provided by the splitter task;
- and
- a joiner task combining the processed data chunks dependent on the control items on the controller queue to produce the output data stream.
A method for integrating task and data parallelism for a dynamic application including at least one task for real-time processing of continuous input data stream to produce an output data stream, comprising the steps of:
- splitting the input data stream into a plurality of individually marked data chunks by a splitter;
- storing the data chunks in a work queue;
- processing subsets of the data chunks by a variable number of worker tasks, each worker task being an instance of the at least one task;
- storing a control item for each data chunk in a controller queue, the control item including task and a data strategy provided by the splitter task;
- and
- combining the processed data chunks by a joiner task dependent on the control items on the controller queue to produce the output data stream.
Dependencies are inferred from the claim text.