Providing strong ordering in multi-stage streaming processing
Assigneesalesforce.com, inc.
InventorsElden Bishop, Jeffrey Chao
Life of this patent
Estimating the term…
Loading patent…
Assigneesalesforce.com, inc.
InventorsElden Bishop, Jeffrey Chao
Estimating the term…
The technology disclosed relates to providing strong ordering in multi-stage processing of near real-time (NRT) data streams. In particular, it relates to maintaining current batch-stage information for a batch at a grid-scheduler in communication with a grid-coordinator that controls dispatch of batch-units to the physical threads for a batch-stage. This includes operating a computing grid, and queuing data from the NRT data streams as batches in pipelines for processing over multiple stages in the computing grid. Also included is determining, for a current batch-stage, batch-units pending dispatch, in response to receiving the current batch-stage information; identifying physical threads that processed batch-units for a previous batch-stage on which the current batch-stage depends and have registered pending tasks for the current batch-stage; and dispatching the batch-units for the current batch-stage to the identified physical threads subsequent to complete processing of the batch-units for the previous batch-stage.
A method, comprising:
A system, comprising:
A non-transitory computer-readable device having instructions stored thereon that, when executed by at least one computing device, causes the at least one computing device to perform operations comprising:
Dependencies are inferred from the claim text.