Graphical management of big data pipelines

A non-transitory computer-readable storage medium tangibly encoded with computer-executable instructions, that when executed by a processor of computing device, performs a method, the method comprising:

• generating, in a database, a first configurable big data processing namespace grouping together a first plurality of big data processing tasks of a big data processing;
• generating, in the database, a second configurable big data processing namespace grouping together a second plurality of big data processing tasks of the big data processing, the second configurable big data processing namespace being generated as a child big data processing namespace of the first configurable big data processing namespace such that the first configurable big data processing namespace is a parent big data processing namespace of the second configurable big data processing namespace;
• generating, in the database, a third configurable big data processing namespace grouping together a third plurality of big data processing tasks of the big data processing, the third configurable big data processing namespace being generated as a child big data processing namespace of the first configurable big data processing namespace such that the first configurable big data processing namespace is a parent big data processing namespace of the third configurable big data processing namespace and the second configurable big data processing namespace is a sibling big data processing namespace of the third configurable big data processing namespace;
• configuring a display scheme for displaying error handling and root cause analysis information in connection with any big data processing task of the big data processing failing to successfully complete execution;
• generating, based partially on the display scheme and using stored information from the database, a graphical user interface (GUI) comprising a navigable hierarchal view with a plurality of components and a plurality of hierarchical levels, the plurality of hierarchical levels comprising namespace and task hierarchical levels, the plurality of components comprising a number of namespace components corresponding to the namespace hierarchical level, a first namespace component corresponding to the first configurable big data processing namespace being displayed, in the navigable hierarchical view, as a parent of namespace components corresponding to the second and third configurable big data processing namespaces, the namespace components corresponding to the first, second and third configurable big data processing namespaces for graphically monitoring and managing big data processing task execution at the namespace hierarchical level, a respective namespace component being selectable to expand the GUI display to include task components for graphically monitoring and managing, at the task hierarchical level, execution of the big data processing tasks of a respective one of the first, second and third big data processing namespaces corresponding to the respective namespace component, the GUI displaying, in connection with a selected component of the plurality of components displayed in the navigable hierarchical view, selectable parts for controlling big data processing task execution, at a hierarchical level of the plurality of hierarchical levels, corresponding to the selected component, the selectable parts for starting, suspending, resuming, and stopping execution of one or more big data processing tasks corresponding to the selected component of the GUI;
• and
• responsive to a change in execution state of big data processing of at least one task of the big data processing, updating one or more of the namespace components in the GUI to indicate the change in execution state of the at least one big data processing task.

A system, comprising:

• a computing device, comprising a processor and a non-transitory computer-readable storage medium for tangibly storing thereon program logic for execution by the processor, the program logic comprising:
• executable logic for generating, in a database, a first configurable big data processing namespace grouping together a first plurality of big data processing tasks of a big data processing;
• executable logic for generating, in the database, a second configurable big data processing namespace grouping together a second plurality of big data processing tasks of the big data processing, the second configurable big data processing namespace being generated as a child big data processing namespace of the first configurable big data processing namespace such that the first configurable big data processing namespace is a parent big data processing namespace of the second configurable big data processing namespace;
• executable logic for generating, in the database, a third configurable big data processing namespace grouping together a third plurality of big data processing tasks of the big data processing, the third configurable big data processing namespace being generated as a child big data processing namespace of the first configurable big data processing namespace such that the first configurable namespace is a parent big data processing namespace of the third configurable big data processing namespace and the second configurable big data processing namespace is a sibling big data processing namespace of the third configurable big data processing namespace;
• executable logic for configuring a display scheme for displaying error handling and root cause analysis information in connection with any big data processing tasks of the big data processing failing to successfully complete execution;
• executable logic for generating, based partially on the display scheme and using stored information from the database, a graphical user interface (GUI) comprising a navigable hierarchal view with a plurality of components and a plurality of hierarchical levels, the plurality of hierarchical levels comprising namespace and task hierarchical levels, the plurality of components comprising a number of namespace components corresponding to the namespace hierarchical level, a first namespace component corresponding to the first configurable big data processing namespace being displayed, in the navigable hierarchical view, as a parent of namespace components corresponding to the second and third configurable big data processing namespaces, the namespace components corresponding to the first, second and third big data processing namespaces for graphically monitoring and managing big data process task execution at the namespace hierarchical level, a respective namespace component being selectable to expand the GUI display to include task components for graphically monitoring and managing, at the task hierarchical level, execution of the big data processing tasks of a respective one of the first, second and third big data processing namespaces corresponding to the respective namespace component, the GUI displaying, in connection with a selected component of the plurality of components in the navigable view, selectable parts for controlling big data processing task execution at a hierarchical level, of the plurality of hierarchical levels, corresponding to the selected component from the GUI, the selectable parts for starting, suspending, resuming, and stopping execution of one or more big data processing tasks corresponding to the selected component of the GUI;
• and
• responsive to a change in execution state of big data processing of at least one task of the big data processing, executable logic for updating one or more of the namespace components in the GUI to indicate the change in execution state of at least one big data processing task.