Architecture for cloud-based data analytics system
AssigneeAlteryx, Inc.
InventorsStephen Gilbert Ahlgren, William Andrew Neal, Chad Robert Auld
Life of this patent
Estimating the term…
Abstract
The present patent application discloses a data analytics system with a split plane architecture that utilizes a workflow management module in a control plane for cloud-based computing. The system receives workflow files from client devices and stores those workflow files in a cloud storage associated with the user. The data plane, which executes the workflow file, retrieves the file directly from the cloud storage or through an API provided by the workflow management module. A workflow engine wrapper, containing a stager system and an execution system, prepares workflow files for execution and stores generated results in the cloud storage.
Claims20
A method comprising:
- receiving, at a control plane of a data analytics system, a request from a client device of a user of the data analytics system to execute a workflow from a packaged workflow file, wherein the packaged workflow file includes at least a workflow for execution, the packaged workflow file stored in a cloud storage account;
- publishing a message to a message queue, the message requesting a job to execute the workflow and job metadata identifying the packaged workflow file;
- responsive to consuming the message, creating, by an execution service in the data plane, an execution engine wrapper container within the data plane, the execution engine wrapper container housing an execution engine;
- retrieving, by the execution engine wrapper container, the packaged workflow file for execution using the job metadata;
- executing the workflow file using the execution engine to generate results of the workflow;
- responsive to determining that the job has been completed, deleting, by the execution service, the execution engine wrapper container;
- and
- transmitting the results of executing the workflow file to the client device.
A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform operations comprising:
- receiving, at a control plane of a data analytics system, a request from a client device of a user of the data analytics system to execute a workflow from a packaged workflow file, wherein the packaged workflow file includes at least a workflow for execution, the packaged workflow file stored in a cloud storage account;
- publishing a message to a message queue, the message requesting a job to execute the workflow and job metadata identifying the packaged workflow file;
- responsive to consuming the message, creating, by an execution service in the data plane, an execution engine wrapper container within the data plane, the execution engine wrapper container housing an execution engine;
- retrieving, by the execution engine wrapper container, the packaged workflow file for execution using the job metadata;
- executing the workflow file using the execution engine to generate results of the workflow;
- responsive to determining that the job has been completed, deleting, by the execution service, the execution engine wrapper container;
- and
- transmitting the results of executing the workflow file to the client device.
A system comprising:
- a processor;
- and
- a non-transitory computer-readable medium storing instructions that, when executed by the processor, cause the processor to perform operations comprising:
- receiving, at a control plane of a data analytics system, a request from a client device of a user of the data analytics system to execute a workflow from a packaged workflow file, wherein the packaged workflow file includes at least a workflow for execution, the packaged workflow file stored in a cloud storage account;
- publishing a message to a message queue, the message requesting a job to execute the workflow and job metadata identifying the packaged workflow file;
- responsive to consuming the message, creating, by an execution service in the data plane, an execution engine wrapper container within the data plane, the execution engine wrapper container housing an execution engine;
- retrieving, by the execution engine wrapper container, the packaged workflow file for execution using the job metadata;
- executing the workflow file using the execution engine to generate results of the workflow;
- responsive to determining that the job has been completed, deleting, by the execution service, the execution engine wrapper container;
- and
- transmitting the results of executing the workflow file to the client device.
Dependencies are inferred from the claim text.