Auto-discovery of data lineage in large computer systems

A computerized method comprising:

• identifying, in a computer system comprising a plurality of different data sources and a plurality of different applications performing operations on data from the plurality of different data sources, one or more processes that transform data originating in one or more of the plurality of different data sources;
• for each of the identified one or more processes, creating a relationship table, the relationship table listing objects input to the process, objects output from the process, and one or more mappings defining transformations performed on the objects input to the process to product the objects output from the process;
• creating one or more dependencies between relationship tables by linking at least an instance of a first object in input to a process in a first relationship table with at least an instance of the first object in output from a process in a second relationship table;
• and
• tracing lineage of an object in the computer system by accessing one or more of the relationship tables and the one or more dependencies between the relationship tables.

A non-transitory machine-readable storage medium comprising instructions, which when implemented by one or more machines, cause the one or more machines to perform operations comprising:

• identifying, in a computer system comprising a plurality of different data sources and a plurality of different applications performing operations on data from the plurality of different data sources, one or more processes that transform data originating in one or more of the plurality of different data sources;
• for each of the identified one or more processes creating a relationship table, the relationship table listing objects input to the process, objects output from the process, and one or more mappings defining transformations performed on the objects input to the process to product the objects output from the process;
• creating one or more dependencies between relationship tables by linking at least an instance of a first object in input to a process in a first relationship table with at least an instance of the first object in output from a process in a second relationship table;
• and
• tracing lineage of an object in the computer system by accessing one or more of the relationship tables and the one or more dependencies between the relationship tables.