US 11,537,434 B2 · System and method of providing system jobs within a compute environment

Abstract

The disclosure relates to systems, methods and computer-readable media for using system jobs for performing actions outside the constraints of batch compute jobs submitted to a compute environment such as a cluster or a grid. The method for modifying a compute environment from a system job disclosure associating a system job to a queuable object, triggering the system job based on an event and performing arbitrary actions on resources outside of compute nodes in the compute environment. The queuable objects include objects such as batch compute jobs or job reservations. The events that trigger the system job may be time driven, such as ten minutes prior to completion of the batch compute job, or dependent on other actions associated with other system jobs. The system jobs may be utilized also to perform rolling maintenance on a node by node basis.

Claims36

Dependent claims

A non-transitory computer-readable medium storing instructions for managing a multi-node compute environment having a computerized scheduler process associated therewith, the instructions when executed by a processor of a computerized device, performs operations comprising:

receiving a submission of at least one workload to be processed by the multi-node compute environment, the submission also comprising data indicating a particular quality of service (QoS) associated with the processing of the at least one workload;
based at least on the submission, creating least one software process, the at least one software process associated with the at least one workload;
identifying at least one resource necessary for processing of the at least one workload according to the particular QoS;
determining that the at least one resource has been made available for processing of the at least one workload;
and
performing via the at least one software process and based on the determination, of one or more configuration actions that are outside of a compute space allocated by the computerized scheduler process,
wherein:
the performance of the one or more configuration actions that are outside the compute space allocated by the computerized scheduler process comprises performance of at least one of (i) a data stage-in operation, or (ii) a data stage-out operation;
and
the instructions are further configured to, when executed, cause execution of at least one of the at least one workload according to the QoS using at least one of (i) data staged-in via the data stage-in operation, or (ii) data staged-out via the data stage-out operation, respectively.

7 more elements

receiving a submission of at least one workload to be processed by the multi-node compute environment, the submission requiring a particular quality of service (QoS) for the processing of the at least one workload;
based at least on the submission, creating at least one software process, the at least one software process associated with the at least one workload;
configuring, via the at least one software process, of at least one resource necessary for processing of the at least one workload according to the particular QoS;
and
based at least on completion of the configuring of the at least one resource, performing via the at least one software process, of at least part of the processing of the at least one workload;
wherein the configuring comprises at least one of (i) staging data in, or (ii) staging data out;
and
wherein the performing via the at least one software process, of the at least part of the processing of the at least one workload, comprises execution of at least part of the at least one workload according to the QoS and using the at least one of the (i) the staged-in data, or (ii) the staged-out data.

5 more elements

creating at least one software process, the at least one software process configured to be associated with one or more queueable objects operative within the multi-node compute environment, the at least one software process comprising an event policy;
monitoring of at least one operational aspect of the multi-node compute environment;
detecting, based at least on the monitoring, of at least one triggering event associated with the event policy;
and
performing via the at least one software process and based on the detection, of one or more file system backup actions,
wherein the at least one software process comprises at least one software object that is configured to submit a series of system jobs to the computerized scheduler process, the series of system jobs comprising (i) a first system job configured to cause allocation of one or more resources associated with the multi-node compute environment for use in the performance of the one or more file system backup actions, (ii) a second system job configured to cause performance of the one or more file system backup actions using at least part of the allocated one or more resources, and (iii) a third system job configured to cause verification of completion of the one or more file system backup actions, and based at least on the verification, cause release of the allocated one or more resources for use by user workload.

3 more elements

A method of managing a multi-node compute environment having a computerized scheduler process associated therewith, the method comprising:

receiving a submission of one or more workloads to be processed by the multi-node compute environment, the submission requiring a particular quality of service (QoS) for the processing of at least one of the one or more workloads;
based at least on the submission, creating at least one software process, the at least one software process associated with the at least one of the one or more workloads;
performing, via the at least one software process, at least one of (i) staging data in, or (ii) staging data out, the at least one of the (i) staging data in, or (ii) staging data out, being necessary for processing of the at least one of the one or more workloads according to the particular QoS;
and
based at least on completion of the at least one of the (i) staging data in, or (ii) staging data out, performing via the at least one software process, execution of the at least one of the one or more workloads according to the QoS using the at least one of (i) the staged-in data, or (ii) the staged-out data.

2 more elements

Dependencies are inferred from the claim text.

System and method of providing system jobs within a compute environment

Life of this patent

System and method of providing system jobs within a compute environment

Life of this patent

Abstract

Claims36