Congestion control for automatic compute capacity saturation

A congestion control system that increases utilization of compute resources in a shared resource pool comprising, the congestion control system comprising:

• a quota manager stored in memory and executable to:
• receive, from a tenant to the shared resource pool, a request for processing of a workload by a transformer model;
• transmit multiple lease requests to a quota service on behalf of the tenant, the multiple lease requests being associated with processing tasks of the workload and specifying quantities of cloud-based resources requested from the shared resource pool associated with the transformer model;
• observe feedback signals from the quota service for a time interval, the feedback signals each tenant to exceed a resource quota limit allocated to the tenant in association with the shared resource pool;
• and
• in response to determining that the feedback signals satisfy overload criteria, dynamically decrease task parallelism of the workload being processed by the cloud-based resources on behalf of the tenant.

One or more tangible computer-readable storage media encoding computer-executable instructions for executing a computer process that increases utilization of compute resources in a shared resource pool comprising, the computer process comprising:

• transmitting lease requests to a quota service on behalf of a tenant to the shared resource pool, the lease requests being associated with various processing tasks and specifying quantities of cloud-based resources requested from the shared resource pool;
• observing feedback signals from the quota service for a time interval, the feedback signals each indicating whether grant of a corresponding one of the lease requests would cause the tenant to exceed a resource quota limit allocated to the tenant;
• and
• based on the feedback signals satisfying overload criteria, dynamically decreasing parallelism of active tasks being processed by the cloud-based resources on behalf of the tenant.