Guided creation interface for streaming data processing pipelines
AssigneeSplunk Inc.
InventorsJared Breeden, Asmita Puri
Life of this patent
Estimating the term…
Abstract
Systems and methods are disclosed for implementing a data processing workflow user interface for a streaming data processing system. The workflow is visually represented as a series of modules along with interconnections for the modules. Each module represents an operation on a streaming data object, such as a data transformation. The user interface enables selection of a workflow template based on a user-specified data source, and then allows the user to customize the workflow template by specifying additional operations to apply to data objects. The interface may show the user a preview of output data objects processed according to the customized workflow.
Claims30
A computer-implemented method comprising:
- receiving, at a streaming data processing system, a first input via a first user interface, the first input corresponding to selection of a data source category, wherein the data source category is associated with a template describing steps for processing data objects from data sources associated with the data source category;
- receiving a second input via a second user interface, the second input corresponding to configuration data for a particular data source, wherein the particular data source is associated with the data source category;
- generating a streaming data pipeline for the particular data source using the template and the configuration data;
- and
- processing a data object from the particular data source using the streaming data pipeline, wherein, upon processing the data object, the streaming data pipeline transforms the data object from a first format associated with the data source category to a second format associated with the streaming data processing system.
A system comprising:
- a data store including computer-executable instructions;
- and
- a processor in communication with the data store, wherein execution of the computer-executable instructions by the processor causes the system to perform operations including:
- receiving, at a streaming data processing system, a first input via a first user interface, the first input corresponding to selection of a data source category, wherein the data source category is associated with a template describing steps for processing data objects from data sources associated with the data source category;
- receiving a second input via a second user interface, the second input corresponding to configuration data for a particular data source, wherein the particular data source is associated with the data source category;
- generating a streaming data pipeline for the particular data source using the template and the configuration data;
- and
- processing a data object from the particular data source using the streaming data pipeline, wherein, upon processing the data object, the streaming data pipeline transforms the data object from a first format associated with the data source category to a second format associated with the streaming data processing system.
A non-transitory computer-readable medium including computer-executable instructions that, when executed by a computing system, cause the computing system to perform operations including:
- receiving, at a streaming data processing system, a first input via a first user interface, the first input corresponding to selection of a data source category, wherein the data source category is associated with a template describing steps for processing data objects from data sources associated with the data source category;
- receiving a second input via a second user interface, the second input corresponding to configuration data for a particular data source, wherein the particular data source is associated with the data source category;
- generating a streaming data pipeline for the particular data source using the template and the configuration data;
- and
- processing a data object from the particular data source using the streaming data pipeline, wherein, upon processing the data object, the streaming data pipeline transforms the data object from a first format associated with the data source category to a second format associated with the streaming data processing system.
Dependencies are inferred from the claim text.