Dynamic run time allocation of distributed jobs with application specific metrics

A computer implemented method for dynamically changing the allocation of processing units on a multi-nodal computer system comprising the steps of:

• executing an application having a plurality jobs, each with at least one processing unit on a plurality of nodes, where each node has at least one central processing unit (CPU) and a memory;
• executing an application metric plug-in associated with the application that determines an application specific metric that is a measurable objective of the application, wherein the application metric plug-in is initiated by the application, wherein the measurable objective of the application specific metric comprises at least one of:
• production rates, data communication rates, data quality, data availability, data throughput, a result value and a result quality;
• collecting the application specific metrics from the application metric plug-in;
• analyzing the application specific metrics;
• when there is an application specific metric outside the desired range, performing the steps of:
• identifying jobs affecting the application specific metric;
• examining hardware system metrics to identify resources affecting the application specific metric;
• assessing potential job and processing unit relocations in order to determine how to best allocate the jobs on the system;
• dynamically changing the allocation of the processing units on the compute nodes based on the collected metrics and identified resources to optimize system resources and interprocess communications;
• wherein the step of dynamically changing the allocation of the processing units further comprises combining at least two processing units into a job on a single node.