Back to results
US 2023/0401232 A1Published application
Cross-cloud replication of recurrently executing data pipelines
AssigneeSnowflake Inc.
InventorsIstvan Cseri, Dinesh Chandrakant Kulkarni, Mihir Dhananjay Kulkarni, Lanhao Wu, Di Fei Zhang
Life of this patent
Estimating the term…
Abstract
In one aspect, a computer-implemented method includes detecting a committed version of recurrently executed tasks of a first data pipeline on a primary deployment that is hosted on a first cloud service, and replicating the committed version of the recurrently executed tasks to a second data pipeline on a secondary deployment that is hosted on a second cloud service.
Claims20
A computer-implemented method comprising:
- detecting, using one or more processor of server, a resume command only for a first data pipeline on a primary deployment that is hosted on a first cloud service, the first data pipeline comprising a first plurality of computing tasks that are configured for execution on the first cloud service, the resume command indicating an explicit user action that changes to the first data pipeline are complete and that the first data pipeline is execution-ready;
- in response to detecting the resume command for the first data pipeline, detecting a committed version of recurrently executed tasks of the first data pipeline and a more recent uncommitted version of recurrently executed tasks of the first data pipeline;
- in response to detecting the resume command for the first data pipeline, selecting the committed version of recurrently executed tasks of the first data pipeline over the more recent uncommitted version of recurrently executed tasks of the first data pipeline for the resume command;
- and
- replicating the committed version of the recurrently executed tasks to a second data pipeline on a secondary deployment that is hosted on a second cloud service, the second data pipeline comprising a second plurality of computing tasks that are configured for execution on the second cloud service,
- wherein the first data pipeline corresponds to a first deployment and a first user account on the first deployment,
- wherein the second data pipeline corresponds to a second deployment and a second user account on the second deployment.
4 more elements
A computing apparatus comprising:
- a processor;
- and
- a memory storing instructions that, when executed by the processor, configure the apparatus to:
- detect a resume command only for a first data pipeline on a primary deployment that is hosted on a first cloud service, the first data pipeline comprising a first plurality of computing tasks that are configured for execution on the first cloud service, the resume command indicating an explicit user action that changes to the first data pipeline are complete and that the first data pipeline is execution-ready;
- in response to detecting the resume command for the first data pipeline, detect a committed version of recurrently executed tasks of the first data pipeline and a more recent uncommitted version of recurrently executed tasks of the first data pipeline;
- in response to detecting the resume command for the first data pipeline, select the committed version of recurrently executed tasks of the first data pipeline over the more recent uncommitted version of recurrently executed tasks of the first data pipeline for the resume command;
- and
- replicate the committed version of the recurrently executed tasks to a second data pipeline on a secondary deployment that is hosted on a second cloud service, the second data pipeline comprising a second plurality of computing tasks that are configured for execution on the second cloud service,
- wherein the first data pipeline corresponds to a first deployment and a first user account on the first deployment,
- wherein the second data pipeline corresponds to a second deployment and a second user account on the second deployment.
7 more elements
A non-transitory computer-readable storage medium, the computer-readable storage medium including instructions that when executed by a computer, cause the computer to:
- detect a resume command only for a first data pipeline on a primary deployment that is hosted on a first cloud service, the first data pipeline comprising a first plurality of computing tasks that are configured for execution on the first cloud service, the resume command indicating an explicit user action that changes to the first data pipeline are complete and that the first data pipeline is execution-ready;
- in response to detecting the resume command for the first data pipeline, detect a committed version of recurrently executed tasks of the first data pipeline and a more recent uncommitted version of recurrently executed tasks of the first data pipeline;
- in response to detecting the resume command for the first data pipeline, selecting the committed version of recurrently executed tasks of the first data pipeline over the more recent uncommitted version of recurrently executed tasks of the first data pipeline for the resume command;
- and
- replicate the committed version of the recurrently executed tasks to a second data pipeline on a secondary deployment that is hosted on a second cloud service, the second data pipeline comprising a second plurality of computing tasks that are configured for execution on the second cloud service,
- wherein the first data pipeline corresponds to a first deployment and a first user account on the first deployment,
- wherein the second data pipeline corresponds to a second deployment and a second user account on the second deployment.
4 more elements
Dependencies are inferred from the claim text.