Design-time information based on run-time artifacts in transient cloud-based distributed computing clusters

A system comprising:

• one or more processors;
• and
• a memory having instructions stored thereon, which when executed by the one or more processors, cause the system to:
• receive first metadata from a first transient computing cluster, the first transient computing cluster temporarily provisioned to process data according to a first workflow, the first metadata including run-time artifacts generated at a time of processing the data according to the first workflow;
• process the first metadata to generate first design-time information associated with the first workflow, the first design-time information indicative of a design of the first transient computing cluster, the first workflow, and/or one or more data processing jobs included in the first workflow;
• and
• generate relationship data based on the first design-time information so as to obtain an input that adjusts the first workflow.

A non-transitory computer readable medium storing instructions, execution of which by a computer system, cause the computer system to:

• receive first metadata from a first transient computing cluster, the first transient computing cluster temporarily provisioned to process data according to a first workflow, the first metadata including run-time artifacts generated at a time of processing the data according to the first workflow;
• process the first metadata to generate first design-time information associated with the first workflow, the first design-time information indicative of a design of the first transient computing cluster, the first workflow, and/or one or more data processing jobs included in the first workflow;
• and
• generate relationship data based on the first design-time information so as to obtain an input that adjusts the first workflow.