Workload management with data access awareness by aggregating file locality information in a computing cluster

A system for workload management by aggregating locality information for a set of files associated with a workload in a cluster of hosts, from a file level to a level of the set of files, the system comprising:

• a processor executing instructions stored in a memory device, wherein the processor:
• selects a subset of files from the set of files;
• resets a set of storage size counters, wherein each of the storage size counters is assigned to a host in the cluster;
• resets an overall storage size counter;
• scans the files in the subset of the set of files, and for each scanned file:
• retrieves locality information of the file,
• adds the locality information of the file to the storage size counters of the hosts, and
• adds a total size of the file to the overall storage size counter;
• computes an output proportion of the storage size counter of each host from the overall storage size counter;
• and
• schedules the workload on certain hosts in the cluster of hosts according to the output proportion of the storage size counter of each host wherein the certain hosts in the cluster of hosts scheduled to execute the workload are chosen based on the output proportion computed from the locality information of the file associated with the workload.

A computer program product for workload management by aggregating locality information for a set of files associated with a workload in a cluster of hosts, from a file level to a level of the set of files, by a processor, the computer program product embodied on a non-transitory computer-readable storage medium having computer-readable program code portions stored therein, the computer-readable program code portions comprising:

• an executable portion that selects a subset of files from the set of files;
• an executable portion that resets a set of storage size counters, wherein each of the storage size counters is assigned to a host in the cluster;
• an executable portion that resets an overall storage size counter;
• an executable portion that scans the files in the subset of the set of files, and for each scanned file:
• retrieves locality information of the file,
• adds the locality information of the file to the storage size counters of the hosts, and
• adds a total size of the file to the overall storage size counter;
• an executable portion that computes an output proportion of the storage size counter of each host from the overall storage size counter;
• and
• an executable portion that schedules the workload on certain hosts in the cluster of hosts according to the output proportion of the storage size counter of each host wherein the certain hosts in the cluster of hosts scheduled to execute the workload are chosen based on the output proportion computed from the locality information of the file associated with the workload.