Distributing application traffic to servers based on dynamic service response time

A non-transitory computer readable storage medium having computer readable program code embodied therewith for distributing application traffic received by a service gateway from a host to a server of a plurality of servers based on dynamic service response time of the server, the computer readable program code configured to:

• receive a first service request for a service session from the host by the service gateway, the first service request having a service request time;
• relay the first service request from the service gateway to a server of the plurality of servers, the relaying occurring over the service session between the service gateway and the server;
• receive a service response from the server, the service response having a service response time;
• calculate a dynamic service processing time for the first service request as a duration between the service request time and the service response time;
• compare the dynamic service processing time with an expected service processing time to determine whether the dynamic service processing time exceeds the expected service processing time by at least a threshold amount, wherein the expected service processing time is based at least in part on a service attribute of the first service request or a service attribute of the server;
• wherein the expected service processing time is determined by the service gateway and stored in a datastore together with an associated service attribute of a service request or service attribute of the server, the service gateway determining the expected service processing time by:
• comparing the first service request or the server with the service attribute in the datastore;
• and
• if the first service request or the server matches the service attribute in the datastore, retrieving the expected service processing time associated with the matching service attribute from the datastore, wherein the expected service processing time is variable based on the matching service attribute;
• update a server busy indicator for the server in response to the comparing the dynamic service processing time with the expected service processing time, wherein the server busy indicator for the server is maintained at the service gateway;
• receive a second service request from the host;
• check the server busy indicator for the server;
• in response to determining that the server busy indicator indicates that the server is busy, place the second service request in a service request buffer of the service gateway and maintain a connection to the host;
• and
• in response to determining that the server busy indicator indicates that the server is not busy, relay the second service request from the service gateway to the server over the service session between the service gateway and the server.

A system, comprising:

• a server for processing service requests;
• and
• a service gateway comprising a processor and a computer readable storage medium having computer readable program code embodied therewith, wherein when the computer readable program code is executed by the processor, causes the service gateway to:
• receive a first service request from a host for a service session, the service request having a service request time;
• relay the first service request to a server over the service session between the service gateway and the server;
• receive a service response from the server, the service response having a service response time;
• calculate a dynamic service processing time for the first service request from the service request time and the service response time;
• compare the dynamic service processing time with an expected service processing time to determine whether the dynamic service processing time exceeds the expected service processing time by at least a threshold amount, wherein the expected service processing time is based at least in part on the service attribute and stored in a datastore;
• wherein the expected service processing time is determined by the service gateway and stored in the datastore together with an associated service attribute of a service request or service attribute of the server, the service gateway determining the expected service processing time by:
• comparing the first service request or the server with the service attribute in the datastore;
• and
• if the first service request or the server matches the service attribute in the datastore, retrieving the expected service processing time associated with the matching service attribute from the datastore, wherein the expected service processing time is variable based on the matching service attribute;
• update a server busy indicator for the server in response to the comparing the dynamic service processing time with the expected service processing time wherein the server busy indicator for the server is maintained at the service gateway;
• receive a second service request from the host;
• check the server busy indicator for the server;
• in response to determining that the server busy indicator indicates that the server is busy, place the second service request in a service request buffer of the service gateway and maintain a connection to the host;
• and
• in response to determining that the server busy indicator indicates that the server is not busy, relay the second service request from the service gateway to the server over the service session between the service gateway and the server.