Processing machine learning models in a machine learning pipeline using virtual resources and vault-based credentials

An apparatus comprising:

• at least one processing device comprising a processor coupled to a memory;
• the at least one processing device being configured to implement the following steps:
• obtaining a request from a user to process at least one machine learning model (ML) in a given stage of a plurality of stages of an ML pipeline;
• obtaining an ML pipeline template to configure at least one virtual computing resource;
• obtaining a namespace associated with the user, wherein the namespace comprises model code for the at least one ML model, at least one artifact of the at least one ML model and one or more credentials of the user stored in a secure digital vault;
• configuring the at least one virtual computing resource using the ML pipeline template and the at least one artifact of the at least one ML model;
• initiating an execution of at least a portion of the model code for the at least one ML model by the configured at least one virtual computing resource, wherein the execution of the model code for the at least one ML model comprises accessing at least one protected resource using at least one of the one or more credentials of the user obtained from the secure digital vault;
• obtaining one or more results from the execution of the at least the portion of the model code for the at least one ML model by the configured at least one virtual computing resource;
• and
• initiating one or more processing steps based at least in part on at least one of the one or more results.

A non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code when executed by at least one processing device causes the at least one processing device to perform the following steps:

• obtaining a request from a user to process at least one machine learning (ML) model in a given stage of a plurality of stages of an ML pipeline;
• obtaining an ML pipeline template to configure at least one virtual computing resource;
• obtaining a namespace associated with the user, wherein the namespace comprises model code for the at least one ML model, at least one artifact of the at least one ML model and one or more credentials of the user stored in a secure digital vault;
• configuring the at least one virtual computing resource using the ML pipeline template and the at least one artifact of the at least one ML model;
• initiating an execution of at least a portion of the model code for the at least one ML model by the configured at least one virtual computing resource, wherein the execution of the model code for the at least one ML model comprises accessing at least one protected resource using at least one of the one or more credentials of the user obtained from the secure digital vault;
• obtaining one or more results from the execution of the at least the portion of the model code for the at least one ML model by the configured at least one virtual computing resource;
• and
• initiating one or more processing steps based at least in part on at least one of the one or more results.