Techniques for replication-aware resource management and task management of file systems

A non-transitory computer-readable medium storing computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:

• enqueuing a plurality of replication jobs into a queue in a region comprising one or more connected data centers, each replication job of the plurality of replication jobs being used for a cross-region replication;
• associating a set of information with each replication job, the set of information comprising a requested processing capacity, priority, and weightage for each replication job;
• allocating resources in the region to a fleet of components, comprising a first component and a second component, for executing the plurality of replication jobs, the first component being a first logical partition of the resources in the region, and the second component being a second logical partition of the resources in the region;
• selecting a first replication job of the plurality of replication jobs for execution from the queue by the first component based at least in part on the requested processing capacity, the priority, and the weightage associated with the first replication job;
• and
• selecting a second replication job of the plurality of replication jobs for execution from the queue by the second component based at least in part on the requested processing capacity, the priority, and the weightage associated with the second replication job, the second component being configured to operate independent of and in parallel with the first component.

A system, comprising:

• one or more processors;
• one or more non-transitory computer readable media storing computer-executable instructions that, when executed by the one or more processors of a computing system, cause the system to:
• enqueue, by a computing system, a plurality of replication jobs into a queue in a region comprising one or more connected data centers, each replication job of the plurality of replication jobs being used for a cross-region replication;
• associate, by the computing system, a set of information with each replication job, the set of information comprising a requested processing capacity, priority, and weightage for each replication job;
• allocate, by the computing system, resources in the region to a fleet of components, comprising a first component and a second component, for executing the plurality of replication jobs, the first component being a first logical partition of the resources in the region, and the second component being a second logical partition of the resources in the region;
• select, by the computing system, a first replication job of the plurality of replication jobs for execution from the queue by the first component based at least in part on the requested processing capacity, the priority, and the weightage associated with the first replication job;
• and
• select, by the computing system, a second replication job of the plurality of replication jobs for execution from the queue by the second component based at least in part on the requested processing capacity, the priority, and the weightage associated with the second replication job, the second component being configured to operate independent of and in parallel with the first component.