Systems and methods for generating data lineage

A computer-implemented method, comprising:

• obtaining domain-specific code in response to a build request, wherein the domain-specific code includes an instance of a domain-specific language configured for a specific domain;
• generating process code, including (i) converting the domain-specific code to a format of the process code and (ii automatically inserting at least one breadcrumb instrumentation element, wherein the at least one breadcrumb instrumentation element is automatically inserted during conversion by a conversion engine, and
• wherein the breadcrumb instrumentation element is not defined in the domain-specific code;
• executing the process code, wherein execution of the process code includes:
• execution of one or more activities defined by the domain-specific code;
• and
• generation of a lineage data structure, via execution of the at least one breadcrumb instrumentation element, that stores data elements representative of a mapping between an input data element and an output data element;
• storing the lineage data structure in a data store;
• and
• implementing a data lineage review process,
• wherein the at least one breadcrumb instrumentation element is inserted in the process code at a location corresponding to a function type defined in the domain-specific code.

A non-transitory computer readable medium having instructions stored thereon, wherein the instructions, when executed by at least one processor, cause at least one device to perform operations comprising:

• obtaining domain-specific code in response to a build request, wherein the domain-specific code includes an instance of a domain-specific language configured for a specific domain;
• generating process code, including (i) converting the domain-specific code to a format of the process code and (ii) automatically inserting at least one breadcrumb instrumentation element, wherein the at least one breadcrumb instrumentation element is automatically inserted during conversion by a conversion engine, and
• wherein the breadcrumb instrumentation element is not defined in the domain-specific code;
• executing the process code, wherein execution of the process code includes:
• execution of one or more activities defined by the domain-specific code;
• and
• generation of a lineage data structure, via execution of the at least one breadcrumb instrumentation element, that stores data elements representative of a mapping between an input data element and an output data element;
• storing the lineage data structure in a data store;
• and
• implementing a data lineage review process,
• wherein the lineage data structure comprises a tuple defining a set of inputs, a set of outputs, and a corresponding activity of the one or more activities defined by the domain-specific code.