System and method for data transformation using dataflow graphs

A non-transitory computer-readable medium having a system of managing data, wherein the system utilizes one or more processors and a single operating system in a multi-thread environment, and wherein the system comprises:

• a plurality of map components, each map component having one or more ports for accepting data and for producing data and each map component encapsulating a particular dataflow pattern;
• compiler tools for organizing and linking said map components using said ports into a dataflow application, said compiler tools, including a map synthesizer which validates links between map components and prepares an execution plan;
• and
• an executor for creating and managing data communication among map components in the dataflow application and executing the dataflow application on said one or more processors in parallel using said execution plan, wherein said executor assigns a separate thread to map processes as a separate thread of execution with data supplied to the system.