Optimizing execution of data transformation flows

A method of transforming data, comprising:

• at a computer system having a display, one or more processors, and memory storing one or more programs configured for execution by the one or more processors:
• displaying a user interface that includes a data flow pane and a data pane;
• receiving first user input to build a node/link data transformation flow diagram in the data flow pane, wherein each node in the flow diagram specifies a respective operation to retrieve data from a respective data source, specifies a respective operation to transform data, or specifies a respective operation to create a respective output data set, and wherein the flow diagram includes a subtree having one or more data source nodes that retrieve data from a first data source and one or more transformation operation nodes;
• receiving a second user input to execute at least the subtree;
• in accordance with the second user input and a determination that the nodes in the subtree are configured to execute imperatively:
• performing the operations of the nodes in the subtree sequentially as specified by links in the subtree, thereby retrieving data from the first data source, transforming the retrieved data, and forming a first intermediate data set;
• and
• displaying the first intermediate data set in the data pane;
• receiving third user input to configure the nodes in the subtree to execute declaratively;
• receiving a fourth user input to execute at least the subtree;
• and
• in accordance with the fourth user input and a determination that the nodes in the subtree are configured to execute declaratively:
• constructing a database query that is logically equivalent to the operations specified by the nodes in the subtree;
• transmitting the database query to the first data source to retrieve a second intermediate data set from the first data source according to the database query;
• and
• displaying the second intermediate data set in the data pane.

A non-transitory computer readable storage medium storing one or more programs configured for execution by a computer system having one or more processors, memory, and a display, the one or more programs comprising instructions for:

• displaying a user interface that includes a data flow pane and a data pane;
• receiving first user input to build a node/link data transformation flow diagram in the data flow pane, wherein each node in the flow diagram specifies a respective operation to retrieve data from a respective data source, specifies a respective operation to transform data, or specifies a respective operation to create a respective output data set, and wherein the flow diagram includes a subtree having one or more data source nodes that retrieve data from a first data source and one or more transformation operation nodes;
• receiving a second user input to execute at least the subtree;
• in accordance with the second user input and a determination that the nodes in the subtree are configured to execute imperatively:
• performing the operations of the nodes in the subtree sequentially as specified by links in the subtree, thereby retrieving data from the first data source, transforming the retrieved data, and forming a first intermediate data set;
• and
• displaying the first intermediate data set in the data pane;
• receiving third user input to configure the nodes in the subtree to execute declaratively;
• receiving a fourth user input to execute at least the subtree;
• and
• in accordance with the fourth user input and a determination that the nodes in the subtree are configured to execute declaratively:
• constructing a database query that is logically equivalent to the operations specified by the nodes in the subtree;
• transmitting the database query to the first data source to retrieve a second intermediate data set from the first data source according to the database query;
• and
• displaying the second intermediate data set in the data pane.