A process for supporting computer infrastructure, said process comprising providing at least one support service for at least one of creating, integrating, hosting, maintaining, and deploying computer-readable code in a computing system, wherein the code in combination with the computing system is capable of performing a method for continuously optimizing data archive management scheduling, said method comprising:
- a job scheduler receiving inputs from an archive management system, wherein the inputs comprise task information, replica placement data, infrastructure topology data, and resource performance data;
- modeling a flow network from the received inputs, such that the flow network represents data content, software programs, physical devices, and communication capacity of the archive management system, and such that an optimal path of the flow network from a task of at least one archive management task to a worker program of the archive management system represents an optimal initial schedule for the worker program to perform the task;
- computing the optimal initial schedule from the optimal path in the flow network;
- transferring the computed optimal initial schedule to the archive management system to perform the optimal initial schedule;
- receiving monitoring results of operations of the transferred optimal initial schedule from the archive management system;
- creating an adjusted schedule as a result of dynamically adjusting the optimal initial schedule based on the received monitoring results;
- and
- transferring the adjusted schedule to the archive management system to perform the adjusted schedule such that the archive management system generates new monitoring results of operations of the transferred adjusted schedule;
- wherein said modeling the flow network comprises:
- creating vertices organized in eight levels comprising LEVEL_0, LEVEL_1, LEVEL_2, LEVEL_3, LEVEL_4, LEVEL_5, LEVEL_6, and LEVEL_7,
- wherein LEVEL_0 comprises a source vertex,
- wherein LEVEL_1 comprises at least one task vertex associated with said at least one the archive management task,
- wherein LEVEL_2 comprises at least one volume vertex associated with a storage device volume necessary to perform said at least one the archive management task,
- wherein LEVEL_3 comprises said at least one volume vertex of LEVEL_2,
- wherein LEVEL_4 comprises at least one server vertex associated with a computing server necessary to perform said at least one the archive management task,
- wherein LEVEL_5 comprises said at least one server vertex of LEVEL_4,
- wherein LEVEL_6 comprises at least one worker vertex associated with at least one worker program that performs said at least one the archive management task, and
- wherein LEVEL_7 comprises a sink vertex;
- creating edges organized in seven types comprising EDGE_01, EDGE_12, EDGE_23, EDGE_34, EDGE_45, EDGE_56, and EDGE_67,
- wherein a first type EDGE_01 comprises a first edge directed from the source vertex of LEVEL_0 to a first task vertex of LEVEL_1, the first edge indicating that a first task associated with the first task vertex is scheduled by the job scheduler,
- wherein a second type EDGE_12 comprises a second edge directed from the first task vertex of LEVEL_1 to a first volume vertex of LEVEL_2, the second edge indicating that a first storage device volume associated with the first volume vertex stores latest data necessary to perform the first task,
- wherein a third type EDGE_23 comprises a third edge directed from the first volume vertex of LEVEL_2 to the first volume vertex of LEVEL_3,
- wherein a fourth type EDGE_34 comprises a fourth edge directed from the first volume vertex of LEVEL_3 to a first server vertex of LEVEL_4, the fourth edge indicating that a first computing server associated with the first server vertex is physically coupled to the first storage device volume,
- wherein a fifth type EDGE_45 comprises a fifth edge directed from the first server vertex of LEVEL_4 to the first server vertex of LEVEL_5,
- wherein a sixth type EDGE_56 comprises a sixth edge directed from the first server vertex of LEVEL_5 to a first worker vertex of LEVEL_6, the sixth edge indicating that the first computing server associated with the first server vertex hosts a first worker program associated with the first worker vertex, and
- wherein a seventh type EDGE_67 comprises a seventh edge directed from the first worker vertex of LEVEL_6 to the sink vertex of LEVEL_7, the seventh edge indicating that the first worker program completes operations scheduled by the job scheduler;
- assigning weights to the created edges,
- wherein the first edge of the first type EDGE_01 has a first weight equal to a time unit to perform the first task,
- wherein the second edge of the second type EDGE_12 has a second weight equal to a volume usage of the first task,
- wherein the third edge of the third type EDGE_23 has a third weight equal to a volume throughput of the first storage device volume indicating a number of tasks that the first storage device volume can simultaneously support,
- wherein the fourth edge of the fourth type EDGE_34 has a fourth weight equal to a capacity of a communication channel between the first storage device volume and the first computing server,
- wherein the fifth edge of the fifth type EDGE_45 has a fifth weight equal to a processing throughput of the first computing server, and
- creating at least one path from the source vertex to the sink vertex following created edges such that the created at least one path represents a respective job schedule for a task in the path by use of at least one storage device volume and at least one computing server in the path to be performed by at least one worker program in the path, wherein the optimal path is a path having a minimum sum of weights of edges in the path.