Optimizing tail latency via workload and resource redundancy in cloud

A computer-implemented method for performing computing processing comprising:

• receiving a plurality of requests for computing processing;
• replicating at least some of the plurality of requests, wherein the requests are replicated based on a fractional replication factor prior to transmission of the plurality of requests;
• transmitting each received request and each replicated request to a computer resource for processing;
• determining a tail latency for performing the requested computing processing;
• incrementally increasing the fractional replication factor when the determined tail latency does not meet a target tail latency;
• and
• increasing computing resources provisioned to perform the processing when the determined tail latency does not meet the target tail latency and increasing the fractional replication factor does not result in a decrease in the determined tail latency.

A computer program product for performing computing processing, the computer program product comprising a non-transitory computer readable storage having program instructions embodied therewith, the program instructions executable by a computer, to cause the computer to perform a method comprising:

• receiving a plurality of requests for computing processing;
• replicating at least some of the plurality of requests, wherein the replicating is repeated for transmission to each of a plurality of computer resources based on a fractional replication factor prior to transmission of the plurality of requests;
• transmitting each received request and each replicated request to a computer resource of the plurality of computer resources for processing;
• determining a tail latency for performing the requested computing processing;
• and
• incrementally increasing the fractional replication factor when the determined tail latency does not meet a target tail latency.

A computer program product for performing computing processing, the computer program product comprising a non-transitory computer readable storage having program instructions embodied therewith, the program instructions executable by a computer, to cause the computer to perform a method comprising:

• receiving a plurality of requests for computing processing;
• replicating at least some of the plurality of requests, wherein the requests are replicated based on a fractional replication factor prior to transmission of the plurality of requests;
• transmitting each received request and each replicated request to a computer resource for processing;
• determining a tail latency for performing the requested computing processing;
• incrementally increasing the fractional replication factor when the determined tail latency does not meet a target tail latency;
• and
• increasing computing resources provisioned to perform the processing when the determined tail latency does not meet the target tail latency and increasing the fractional replication factor does not result in a decrease in the determined tail latency.

A system for performing computing processing, the system comprising a processor, memory accessible by the processor, and computer program instructions stored in the memory and executable by the processor to perform:

• receiving a plurality of requests for computing processing;
• replicating at least some of the plurality of requests, wherein the replicating is repeated for transmission to each of a plurality of computer resources based on a fractional replication factor prior to transmission of the plurality of requests;
• transmitting each received request and each replicated request to a computer resource of the plurality of computer resources for processing;
• determining a tail latency for performing the requested computing processing;
• and
• incrementally increasing the fractional replication factor when the determined tail latency does not meet a target tail latency.

A system for performing computing processing, the system comprising a processor, memory accessible by the processor, and computer program instructions stored in the memory and executable by the processor to perform:

• receiving a plurality of requests for computing processing;
• replicating at least some of the plurality of requests, wherein the requests are replicated based on a fractional replication factor prior to transmission of the plurality of requests;
• transmitting each received request and each replicated request to a computer resource for processing;
• determining a tail latency for performing the requested computing processing;
• incrementally increasing the fractional replication factor when the determined tail latency does not meet a target tail latency;
• and
• increasing computing resources provisioned to perform the processing when the determined tail latency does not meet the target tail latency and increasing the fractional replication factor does not result in a decrease in the determined tail latency.

A system for performing computing processing, comprising:

• a plurality of computing resources adapted to perform computing processing;
• and
• a load replicator adapted to:
• receive a plurality of requests for computing processing,
• replicate at least some of the plurality of requests, wherein the replicating is repeated for transmission to each of a plurality of computer resources based on a fractional replication factor prior to transmission of the plurality of requests,
• transmit each received request and each replicated request to a computer resource of the plurality of computer resources for processing,
• determine a tail latency for performing the requested computing processing;
• and
• incrementally increase the fractional replication factor when the determined tail latency does not meet a target tail latency.