System and method for dynamic, incremental recommendations within real-time visual simulation

A system for providing a software development component for use with a data integration or other computing environment, comprising:

• one or more processors operable to:
• provide a graphical user interface for design or management of a data flow pipeline adapted to:
• access or receive data from one or more data sources that include a first dataset;
• and
• provide the data to one or more data targets that include a second dataset;
• wherein each data flow includes a specification of the one or more data sources and data targets that operate as hubs and comprise datasets having attributes associated therewith,
• wherein a data flow is associated with actions that operate on one or more input datasets to transform and output data to one or more output datasets, and
• wherein a dataflow software application operates to receive input data from a source of data, and publish output data to one or more destinations, according to the data flow associated with the software application;
• determine, by reference to a knowledge source of the system and metadata stored therein and describing the one or more data sources and one or more data targets, one or more data transformations adapted to map the one or more data sources and one or more data targets, and receive as input the data from the first dataset and output the data to the second dataset;
• and
• display, at the graphical user interface, an indication of the one or more transformations, for selection and incorporation into the design or management of the data flow pipeline, during processing of the accessed data.

A non-transitory computer readable storage medium, including instructions stored thereon which when read and executed by one or more computers cause the one or more computers to perform a method comprising:

• providing a graphical user interface for design or management of a data flow pipeline adapted to:
• access or receive data from one or more data sources that include a first dataset;
• and
• provide the data to one or more data targets that include a second dataset;
• wherein each data flow includes a specification of the one or more data sources and data targets that operate as hubs and comprise datasets having attributes associated therewith,
• wherein a data flow is associated with actions that operate on one or more input datasets to transform and output data to one or more output datasets, and
• wherein a dataflow software application operates to receive input data from a source of data, and publish output data to one or more destinations, according to the data flow associated with the software application;
• determining, by reference to a knowledge source of a system and metadata stored therein and describing the one or more data sources and one or more data targets, one or more data transformations adapted to map the one or more data sources and one or more data targets, and receive as input data from the first dataset and output the data to the second dataset;
• and
• displaying, at the graphical user interface, an indication of the one or more transformations, for selection and incorporation into the design or management of the data flow pipeline, during processing of the accessed data.