Server-side control of power based on service-level agreement

A system to throttle at least one GPU of a node based on a power policy limit and a throttling order, the system comprising:

• at least one computer processor;
• and
• computer storage media storing computer-useable instructions that, when used by the at least one computer processor, cause the system to perform operations comprising:
• obtaining, at the node of the system, a first indication indicative of (1) a plurality of tasks for execution and (2) a corresponding user account associated with each task;
• accessing a plurality of computer resource allocation parameters defining a plurality of priority levels, wherein each corresponding user account is associated with a respective priority level, wherein the respective priority levels of the user accounts collectively define the throttling order for a plurality of GPUs to execute the plurality of tasks;
• based on the power policy limit associated with the node, determining that a power consumption associated with the execution of the plurality of tasks exceeds the power policy limit associated with the node;
• and
• based on the power consumption exceeding the power policy limit and based on the throttling order, throttling the at least one GPU of the plurality of GPUs until the power consumption is at or below the power policy limit associated with the node, wherein throttling the at least one GPU comprises at least one of:
• causing a rack manager to distribute power within a rack comprising the node until the power consumption is equal to or less than the power policy limit associated with the node;
• causing an MB BMC to distribute power within the node or among the plurality of GPUs until the power consumption is equal to or less than the power policy limit associated with the node;
• or
• causing a UBB BMC to distribute power among the plurality of GPUs until the power consumption is equal to or less than the power policy limit associated with the node.

A computer-implemented method, comprising:

• obtaining, at a node of a rack, a first indication indicative of (1) a plurality of tasks for execution and (2) a corresponding user account associated with each task;
• accessing at least one service-level agreement (SLA) defining a plurality of priority levels, wherein each corresponding user account is associated with a respective priority level, wherein the respective priority levels of the user accounts collectively define a throttling order for a plurality of coprocessors to execute the plurality of tasks;
• based on a power policy limit associated with the node, determining that a power consumption associated with the execution of the plurality of tasks exceeds the power policy limit associated with the node;
• and
• based on the power consumption exceeding the power policy limit and based on the throttling order, throttling at least one coprocessor of the plurality of coprocessors until the power consumption is at or below the power policy limit associated with the node, wherein throttling the at least one coprocessor comprises shutting down a virtual machine assigned to a first set of tasks of a plurality of tasks, wherein the first set of tasks is associated with a user account having a priority level that is lower than a priority level of a user account associated with a second set of tasks of the plurality of tasks.