US 10,313,248 B2Granted

Data flow node validation

AssigneeAdobe Inc.

InventorDavid M. Tompkins

Life of this patent

Estimating the term…

Loading patent…

Abstract

Data flow node validation and provisioning techniques are described. In one or more implementations, a system is described that supports visual design and deployment of data flow pipelines to process streaming data flows. The system may be configured to include nodes and connections between the nodes to represent an arbitrary execution graph of data science algorithms (as algorithm action components) that are used to process the streaming data flows. The system may also support validation techniques to verify that the data flow pipeline may operate as intended. Further, the system may also support implementation and provisioning techniques that involve estimation and adjustment of runtime resource provisioning of a deployed data flow pipeline without preemption or starvation occurring for nodes within the pipeline.

Claims19

Dependent claims

A method comprising:

receiving a selection, generated via user interaction with a graphical user interface of at least one computing device, indicating an addition to or removal of a node to or from a data flow pipeline as then having a plurality of nodes as a result of the selection, the plurality of nodes of the pipeline configured to process, originate, or receive streaming data that flows from at least one input data source accessible via a network, the data flow pipeline representing an execution graph of algorithm actions performed by the plurality of nodes as implemented by a processing system and non-transitory computer-readable storage media;
obtaining, by the at least one computing device, live samples in real time from the plurality of nodes responsive to the receipt of the selection indicating the addition to or removal of the node to or from the data flow pipeline;
generating, by the at least one computing device, a real-time online estimation of expected data flow rates through the plurality of nodes of the data flow pipeline by processing the obtained live samples;
validating, automatically and without user intervention by the at least one computing device in real time, whether the expected data flow rates through the plurality of nodes of the data flow pipeline comply with predefined supported data flow rates based at least in part on the online estimation;
outputting, by the at least one computing device in response to the validating, a visual indication indicative of the data flow rates through the plurality of nodes of the data flow pipeline in real-time corresponding to the node addition to or node removal from the data flow pipeline;
and
adjusting, by the at least one computing device automatically and without user intervention in real time responsive to the validating, the data flow pipeline by adding another node to or removing at least one of the plurality of nodes from the data flow pipeline.

4 more elements

A system comprising:

at least one processor;
and
at least one computer-readable storage memory device containing at least one module that, responsive to execution by the at least one processor, causes the at least one module to perform operations comprising:
receiving a selection, generated via user interaction with a graphical user interface, indicating an addition to or removal of a node to or from a data flow pipeline being designed as then having a plurality of nodes as a result of the selection, the plurality of nodes of the pipeline configured to process, originate, or receive streaming data from at least one input data source accessible via a network, the plurality of nodes of the data flow pipeline representing an execution graph of algorithm actions implemented by a processing system and non-transitory computer-readable storage media of at least one computing device;
sampling machine performance metrics in real time from the plurality of nodes responsive to the receipt of the selection indicating the addition to or removal of the node to or from the data flow pipeline;
generating a real-time online estimation of expected data flow rates through the plurality of nodes by processing the sampled machine performance metrics;
validating whether the expected data flow rates through the plurality of nodes of the data flow pipeline comply with predefined supported data flow rates based at least in part on the online estimation, the validating performed automatically and without user intervention in real-time;
and
outputting a visual indication indicative of the validated data flow rates in real-time in the graphical user interface.

6 more elements

Dependencies are inferred from the claim text.

Facts

Application: 14/308,492Filed 18 Jun 2014
Published: US 2015/0372877 A124 Dec 2015
Granted: 4 Jun 2019
Term ends: Not estimatedThe source records don’t support an estimate.

Classification8 codes

H04L 47/20
Flow control; Congestion control: traffic policing
G06F 3/04842
For the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a…:…
H04L 41/16
Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks: using machine learning…
H04L 41/5009
Managing SLA; Interaction between SLA and QoS: determining service level performance parameters or violations of service level contracts, e.g.…

Family

US 2015/0372877 A1Pre-grant publication · published 24 Dec 2015
US 10,313,248 B2This patent

From the US filing history. Foreign family members aren’t listed.

Timeline

Citations

Open elsewhere

Building an agent or pulling many patents? Use the API or MCP server instead of scraping pages. Get an API key