Dilation of sub-flow operators in a data flow

A computer implemented method for generating a data flow for an application, the computer implemented method comprising:

• executing a code generation application on a processor, wherein the code generation application configures the processor for:
• receiving a user-composed data flow of high-level operations, the user composing the data flow by selecting one or more heterogeneous sub-flows through a graphical user interface;
• dilating the data flow to integrate the one or more heterogeneous sub-flows, comprising:
• traversing the data flow to locate sub-flow operators, wherein each sub-flow operator is associated with a corresponding sub-flow, each sub-flow itself comprising a set of one or more operators, and wherein each sub-flow operator is bound to other operators in the data flow through at least one input port of the sub-flow operator and at least one output port of the sub-flow operator;
• for each located sub-flow operator, retrieving the sub-flow corresponding to the sub-flow operator;
• injecting the corresponding sub-flows into the data flow, wherein injecting each sub-flow comprises:
• copying each operator in the sub-flow,
• renaming each operator in the sub-flow, to form renamed operators, and
• inserting the renamed operators into the data flow, thereby replacing the sub-flow operator in the data flow;
• bridging at least the one input port and the one output port of the sub-flow operator to ports of operators in the data flow;
• and
• sending the data flow for code generation, wherein the code generation comprises generating application source code representing the work-flow.

A computer readable storage medium containing a program which, when executed by a computer processor, performs an operation, comprising:

• executing a code generation application on a processor, wherein the code generation application configures the processor for:
• traversing the first data flow to locate sub-flow operators, wherein each sub-flow operator is associated with a corresponding sub-flow, each sub-flow itself comprises a set of one or more operators, and wherein each sub-flow operator is bound to other operators in the first data flow through at least one input port of the sub-flow operator and at least one output port of the sub-flow operator;
• for each located sub-flow operator, retrieving the sub-flow corresponding to the sub-flow operator;
• injecting the corresponding sub-flows into the first data flow, wherein injecting each sub-flow comprises:
• copying each operator in the sub-flow,
• renaming each operator in the sub-flow, to form renamed operators, and
• inserting the renamed operators into the first data flow, thereby replacing the sub-flow operator in the first data flow;
• bridging at least the one input port and the one output port of the sub-flow operator to ports of operators in the first data flow;
• and
• sending the first data flow for code generation, wherein the code generation comprises generating application source code representing the work flow.

A system, comprising:

• a processor;
• and
• a memory containing a code generation application for execution on the processor, wherein the code generation application configures the processor for:
• traversing the first data flow to locate sub-flow operators, wherein each sub-flow operator is associated with a corresponding sub-flow, each sub-flow itself comprises a set of one or more operators, and wherein each sub-flow operator is bound to other operators in the first data flow through at least one input port of the sub-flow operator and at least one output port of the sub-flow operator,
• for each located sub-flow operator, retrieving the sub-flow corresponding to the sub-flow operator,
• injecting the corresponding sub-flows into the first data flow, wherein injecting each sub-flow comprises:
• copying each operator in the sub-flow;
• renaming each operator in the sub-flow, to form renamed operators;
• and
• inserting the renamed operators into the first data flow, thereby
• replacing the sub-flow operator in the first data flow,
• bridging at least the one input port and the one output port of the sub-flow operator to ports of operators in the first data flow, and
• sending the first data flow for code generation, wherein the code generation comprises generating application source code representing the work flow.