Scalable data transfer in and out of analytics clusters

A computer program product for directing transactions within an analytics cluster, the computer program product comprising a computer readable storage device having program code embodied therewith, the program code executable by a processor to:

• support read and write requests in and out of an analytics cluster, the analytics cluster including a distributed storage layout with a plurality of nodes separated into regions, at least one head node in communication with at least one compute node in each region, and each region having a hardware element in communication with the at least one head node;
• store request routing information in the regional hardware element;
• direct data to support communication to one of the plurality of nodes in one of the regions through the regional hardware element, the direct in response to a directive from at least one head node, wherein the direct distributes the data to the cluster, including accounting for requirements of a supporting application while mitigating resource consumption to support a write request and providing direction to the regional hardware element of a select region to access an I/O request to support a read request;
• and
• transfer data responsive to the data direction, wherein the transfer is direct to a compute node in the select region to support the write request and the transfer is a direction through the regional hardware element in the select region to support the read request.

A system comprising:

• a plurality of nodes separated into regions in an analytics cluster, the analytics cluster including a distributed storage layout with a plurality of nodes separated into regions, at least one head node in communication with at least one compute node in each region, and each region having a hardware element in communication with the at least one head node;
• a storage manager to store request routing information in the regional hardware element;
• a direction manager to support communication to one of the plurality of nodes in one of the regions through the regional hardware element, wherein the direction manager distributes the data to the cluster, including accounting for requirements of a supporting application while mitigating resource consumption to support a write request and providing direction to the regional hardware element of a select region to access an I/O request to support a read request;
• and
• a transfer manager to transfer data responsive to the data direction, wherein the transfer is direct to a compute node in the select region to support the write request and the transfer is a direction through the regional hardware element in the select region to support the read request.