Service management using dynamically calculated requests per second thresholds

A server computing device comprising:

• a plurality of hardware resources;
• a first plurality of service instances of a first service executing on the plurality of hardware resources;
• a resource usage tracker executing on the server computing device and configured to perform first operations comprising:
• obtaining, for over a first span of time and across the first plurality of service instances of the first service, a first resource usage of a hardware resource of the plurality of hardware resources and first requests per second (RPS) by the first plurality of service instances to obtain a first plurality of service instance data points comprising a first plurality of resource usage metrics related to a first plurality of RPS numbers;
• performing a first regression of the first plurality of service instance data points to obtain a first current set of correlation functions between the first resource usage and the first RPS, wherein the first current set of correlation functions comprises a high correlation function, a mid-tier correlation function, and a low correlation function, and wherein the first RPS threshold determined from the high correlation function, and
• selecting a first RPS threshold according to the high correlation function and a first current amount of the hardware resource assigned to the first service,
• selecting a second RPS threshold according to the mid-tier correlation function and the first current amount of the hardware resource assigned to the first service;
• selecting a third RPS threshold according to the low correlation function and the first current amount of the hardware resource assigned to the first service;
• and
• a service manager executing on the server computing device and configured to perform second operations comprising:
• continually monitoring the first RPS for the first service as a proxy for hardware resource usage to detect when the first RPS satisfies the first RPS threshold, and
• changing executing of the first service responsive to the first RPS satisfying the first RPS threshold,
• wherein the first RPS threshold, the second RPS threshold, and the third RPS threshold are used to performed independent management of the first service.

A non-transitory computer readable medium comprising computer readable program code for causing a computing device to perform operations, the operations comprising:

• monitoring, over a first span of time and across a first plurality of service instances of a first service, a first resource usage of a hardware resource and first requests per second (RPS) by the first plurality of service instances to obtain a first plurality of service instance data points comprising a first plurality of resource usage metrics related to a first plurality of RPS numbers;
• performing a first regression of the first plurality of service instance data points to obtain a first current set of correlation functions between the first resource usage and the first RPS, wherein the first current set of correlation functions comprises a high correlation function, a mid-tier correlation function, and a low correlation function, and wherein the first RPS threshold determined from the high correlation function;
• selecting a first RPS threshold according to the high correlation function and a first current amount of the hardware resource assigned to the first service;
• selecting a second RPS threshold according to the mid-tier correlation function and the first current amount of the hardware resource assigned to the first service;
• selecting a third RPS threshold according to the low correlation function and the first current amount of the hardware resource assigned to the first service;
• continually monitoring the first RPS for the first service as a proxy for hardware resource usage to detect when the first RPS satisfies the first RPS threshold;
• and
• changing executing of the first service responsive to the first RPS satisfying the first RPS threshold,
• wherein the first RPS threshold, the second RPS threshold, and the third RPS threshold are used to performed independent management of the first service.