Backup from a kubernetes cluster using lightweight on-demand in-cluster resources

A computer-implemented method comprising:

• by a first computing device that executes a sponsor data agent, wherein the first computing device comprises one or more hardware processors, and wherein the first computing device operates outside a Kubernetes cluster:
• responsive to a notice of a backup job of primary data that is stored in a first persistent volume of the Kubernetes cluster,
• using a first communicative connection with an Application Programming Interface server (API server) of the Kubernetes cluster to:
• (a) cause the Kubernetes cluster to create a secondary persistent volume that comprises primary data captured in a snapshot of the first persistent volume, (b) cause the Kubernetes cluster to instantiate a worker pod, which is attached to the secondary persistent volume and is configured with computer-executable instructions for a file client and with a cloned security certificate of the sponsor data agent, and (c) cause the worker pod to execute the file client, and
• terminating the first communicative connection to the API server, based on determining that the worker pod is executing the file client,
• by the file client executing at the worker pod in the Kubernetes cluster, wherein the worker pod executes on a second computing device that comprises one or more hardware processors:
• using the cloned security certificate for a communication tunnel between the worker pod and the first computing device,
• identifying, among the primary data in the secondary persistent volume, payload data and payload-related metadata that changed since a reference time, and
• transmitting, to the sponsor data agent, the payload data and the payload-related metadata that changed, by way of a file backup pipeline within the communication tunnel;
• by the sponsor data agent:
• transmitting, to a media agent that executes on the first computing device, backup data resulting from converting the payload data and the payload-related metadata received from the file client into a format that is proprietary to a system comprising the sponsor data agent and the media agent, and
• based on determining that all of the payload data and the payload-related metadata have been received from the file client, using a second communicative connection with the API server to:
• cause the Kubernetes cluster to tear down one or more of:
• the worker pod, the secondary persistent volume, and the snapshot;
• and
• by the media agent:
• generating one or more backup copies based on the backup data received from the sponsor data agent, and
• storing the one or more backup copies at a secondary storage that is configured outside the Kubernetes cluster.

A system comprising:

• a first computing device that operates outside a Kubernetes cluster, wherein the first computing device comprises one or more first hardware processors and non-transitory computer-readable media comprising computer-executable instructions, which, when executed by the one or more first hardware processors, cause the first computing device to:
• receive a notice of a backup job of primary data that is stored in a first persistent volume of the Kubernetes cluster,
• use a first communicative connection with an Application Programming Interface server (API server) of the Kubernetes cluster to:
• (a) cause the Kubernetes cluster to create a secondary persistent volume that comprises primary data captured in a snapshot of the first persistent volume, (b) cause the Kubernetes cluster to instantiate a worker pod, which is attached to the secondary persistent volume and is configured with computer-executable instructions for a file client and with a cloned security certificate of the first computing device, and (c) cause the worker pod to execute the file client, and
• terminate the first communicative connection to the API server, based on determining that the worker pod is executing the file client,
• wherein the worker pod executes on a second computing device that comprises one or more hardware processors, wherein the cloned security certificate is used for a communication tunnel between the worker pod and the first computing device, and wherein the file client executing at the worker pod is configured to:
• identify, among the primary data in the secondary persistent volume, payload data and payload-related metadata that changed since a reference time, and
• transmit the payload data and the payload-related metadata that changed, by way of a file backup pipeline within the communication tunnel, to the first computing device;
• wherein the computer-executable instructions further cause the first computing device to:
• based on determining that all of the payload data and the payload-related metadata have been received from the file client, use a second communicative connection with the API server to cause the Kubernetes cluster to tear down one or more of:
• the worker pod, the secondary persistent volume, and the snapshot, and
• generate one or more backup copies that are based on the payload data and the payload-related metadata received from the file client, and
• store the one or more backup copies at a secondary storage that is configured outside the Kubernetes cluster.