Apparatus and method for managing data stream distributed parallel processing service

An apparatus running on a processor for managing a data stream distributed parallel processing service, comprising:

• a service manager that registers a plurality of tasks constituting the data stream distributed parallel processing service using the processor;
• a Quality of Service (QoS) monitor that gathers information about an aggregate load of the plurality of tasks and information about an individual load of each of a plurality of nodes constituting a cluster which provides the data stream distributed parallel processing service;
• and
• a scheduler that arranges the plurality of tasks by distributing the plurality of tasks among the plurality of nodes based on the information about the aggregate load of the plurality of tasks and the information about the individual load of each of the plurality of nodes,
• wherein the information about the aggregate load of the plurality of tasks comprises information related to task Central Processing Unit (CPU) load and Input/Output (I/O) load between the tasks,
• wherein if I/O overload occurs between the tasks because the I/O load between the tasks is higher than the load of the plurality of nodes included in the information about the individual load of each of the plurality of nodes, the scheduler selects tasks, causing the I/O overload between the tasks, as I/O overload tasks, and rearranges the I/O overload tasks based on the information about the aggregate load of the plurality of tasks and the information about the I/O load between the tasks, and
• wherein the scheduler arranges the I/O overload tasks as separate threads of a single process at a single node.