# US 10,313,248 B2 · Data flow node validation > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,313,248 B2 - Document ID: US10313248B2 - Status: Granted - Kind: B2 - Application number: 14/308,492 - Filed: 2014-06-18 - Published: 2015-12-24 - Granted: 2019-06-04 - Earliest claimed: 2014-06-18 - Assignees: Adobe Inc. - Applicants: Adobe Inc. - Inventors: David M. Tompkins - Web page: https://patentgrep.com/patent/US10313248B2 ## CPC - H04L 47/20: Traffic policing - G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) - H04L 41/16: using machine learning or artificial intelligence - H04L 41/5009: Determining service level performance parameters or violations of service level contracts, e.g. violations of agreed response time or mean time between failures [MTBF] - H04L 41/5025: by proactively reacting to service quality change, e.g. by reconfiguration after service quality degradation or upgrade - H04L 41/5051: Service on demand, e.g. definition and deployment of services in real time - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Data flow node validation and provisioning techniques are described. In one or more implementations, a system is described that supports visual design and deployment of data flow pipelines to process streaming data flows. The system may be configured to include nodes and connections between the nodes to represent an arbitrary execution graph of data science algorithms (as algorithm action components) that are used to process the streaming data flows. The system may also support validation techniques to verify that the data flow pipeline may operate as intended. Further, the system may also support implementation and provisioning techniques that involve estimation and adjustment of runtime resource provisioning of a deployed data flow pipeline without preemption or starvation occurring for nodes within the pipeline. ## Claims ### Claim 1 A method comprising: receiving a selection, generated via user interaction with a graphical user interface of at least one computing device, indicating an addition to or removal of a node to or from a data flow pipeline as then having a plurality of nodes as a result of the selection, the plurality of nodes of the pipeline configured to process, originate, or receive streaming data that flows from at least one input data source accessible via a network, the data flow pipeline representing an execution graph of algorithm actions performed by the plurality of nodes as implemented by a processing system and non-transitory computer-readable storage media; obtaining, by the at least one computing device, live samples in real time from the plurality of nodes responsive to the receipt of the selection indicating the addition to or removal of the node to or from the data flow pipeline; generating, by the at least one computing device, a real-time online estimation of expected data flow rates through the plurality of nodes of the data flow pipeline by processing the obtained live samples; validating, automatically and without user intervention by the at least one computing device in real time, whether the expected data flow rates through the plurality of nodes of the data flow pipeline comply with predefined supported data flow rates based at least in part on the online estimation; outputting, by the at least one computing device in response to the validating, a visual indication indicative of the data flow rates through the plurality of nodes of the data flow pipeline in real-time corresponding to the node addition to or node removal from the data flow pipeline; and adjusting, by the at least one computing device automatically and without user intervention in real time responsive to the validating, the data flow pipeline by adding another node to or removing at least one of the plurality of nodes from the data flow pipeline. ### Claim 2 (depends on 1) The method as described in claim 1, wherein the obtaining, the generating, the validating, the outputting, and the adjusting are repeated for a plurality of iterations. ### Claim 3 (depends on 1) The method as described in claim 1, wherein the node is configured to process the streaming data and thus is representative of algorithm actions to be performed by the node to perform data processing as well as an amount of computing or network resources allocated to perform the algorithm actions of the node. ### Claim 4 (depends on 1) The method as described in claim 1, wherein the validating includes validating that data flow rates through the plurality of nodes of the data flow pipeline do not result in overflow or underflow conditions at individual ones of the plurality of nodes. ### Claim 5 (depends on 4) The method as described in claim 4, wherein the validating is performed using latency constraints. ### Claim 6 (depends on 4) The method as described in claim 4, wherein at least one of the plurality of nodes is configured as a join point or a diversion point in the data flow pipeline. ### Claim 7 (depends on 1) The method as described in claim 1, wherein the validating includes validating the data flow pipeline does not produce runaway acyclic feedback. ### Claim 8 (depends on 1) The method as described in claim 1, wherein the validating is performed by calculating characterization parameters of algorithm action components of the node. ### Claim 9 (depends on 1) The method as described in claim 1, wherein the live samples are measured based on a number of records received for a predefined amount of time or an amount of data received for a predefined amount of time. ### Claim 10 A system comprising: at least one processor; and at least one computer-readable storage memory device containing at least one module that, responsive to execution by the at least one processor, causes the at least one module to perform operations comprising: receiving a selection, generated via user interaction with a graphical user interface, indicating an addition to or removal of a node to or from a data flow pipeline being designed as then having a plurality of nodes as a result of the selection, the plurality of nodes of the pipeline configured to process, originate, or receive streaming data from at least one input data source accessible via a network, the plurality of nodes of the data flow pipeline representing an execution graph of algorithm actions implemented by a processing system and non-transitory computer-readable storage media of at least one computing device; sampling machine performance metrics in real time from the plurality of nodes responsive to the receipt of the selection indicating the addition to or removal of the node to or from the data flow pipeline; generating a real-time online estimation of expected data flow rates through the plurality of nodes by processing the sampled machine performance metrics; validating whether the expected data flow rates through the plurality of nodes of the data flow pipeline comply with predefined supported data flow rates based at least in part on the online estimation, the validating performed automatically and without user intervention in real-time; and outputting a visual indication indicative of the validated data flow rates in real-time in the graphical user interface. ### Claim 11 (depends on 10) The system as described in claim 10, wherein the operations further comprise, responsive to the validating that the data flow rates do not comply with the predefined supported data flow rates, adjusting the data flow pipeline by adding another node to or removing at least one of the plurality of nodes from the data flow pipeline. ### Claim 12 (depends on 11) The system as described in claim 11, wherein the operations further comprise repeating the obtaining, the generating, and the validating responsive to the adjusting. ### Claim 13 (depends on 11) The system as described in claim 11, wherein the node is configured to process the data and thus is representative of algorithm actions to be performed by the node to perform data processing as well as an amount of computing resources allocated to perform the algorithm actions of the node. ### Claim 14 (depends on 10) The system as described in claim 10, wherein the validating includes validating that data flow rates through the plurality of nodes of the data flow pipeline do not result in overflow or underflow conditions at individual ones of the plurality of nodes. ### Claim 15 (depends on 14) The system as described in claim 14, wherein the validating is performed using latency constraints. ### Claim 16 (depends on 10) The system as described in claim 10, wherein the validating operations include validating that the data flow pipeline does not produce runaway acyclic feedback. ### Claim 17 (depends on 10) The system as described in claim 10, wherein the operations further comprise performing the validating by calculating characterization parameters of algorithm action components of the plurality of nodes. ### Claim 18 A system for validating streaming data flow rates, the system comprising: a data flow pipeline implemented as an execution graph of algorithm actions performed to process streaming data flows from at least one input data source, the data flow pipeline implemented by a processing system and non-transitory computer-readable storage media of at least one computing device; a data flow design module implemented by the processing system and non-transitory computer-readable storage media of the at least one computing device to receive a selection via a data flow user interface indicating an addition to or removal of a node to or from the data flow pipeline as then having a plurality of nodes as a result of the selection, a flow rate module implemented by the processing system and non-transitory computer-readable storage media of the at least one computing device to: obtain live samples of machine performance metrics in real time from the plurality of nodes; determine expected data flow rates through the data flow pipeline using the obtained live samples; validate, automatically and without user intervention in real time, whether the expected data flow rates comply with predefined supported data flow rates for the plurality of nodes of the data flow pipeline; and output, via the data flow user interface in real time, a visual indication indicative of a result of the validation. ### Claim 19 (depends on 18) The system as described in claim 18, wherein the visual indication indicates whether the addition to or removal of the node from the data flow pipeline results in overflow or underflow conditions at individual ones of the plurality of nodes.