Dataset identification for datasets with multiple identification attributes

A method for lineage-driven dataset identification, comprising:

• receiving, by a system, information identifying a first dataset, with a first identification attribute,
• wherein the information includes:
• the first identification attribute that identifies the first dataset, and
• other information identifying at least a second dataset linked to the first dataset by a process;
• processing, by the system, the first identification attribute to generate a first identifier for the first dataset;
• searching, by the system, a data lineage based graph representation of a plurality of datasets to identify a graph node representing the first dataset within the data lineage based graph representation of the plurality of datasets;
• updating, by the system, the data lineage based graph representation of the plurality of datasets to link the first dataset with the second dataset based on searching the data lineage based graph representation of the plurality of datasets to identify the graph node;
• processing, by the system, a second identification attribute of the second dataset to generate a second identifier of the second dataset;
• determining, by the system, that the second dataset is not represented in the data lineage based graph representation;
• and
• adding, by the system and based on determining that the second dataset is not represented in the data lineage based graph representation, at least one grouping to a data store, wherein the at least one grouping corresponds to the identifier of the second dataset.

A non-transitory computer-readable medium storing a set of instructions, the set of instructions comprising:

• one or more instructions that, when executed by one or more processors of a system, cause the system to:
• receive information identifying a dataset including a plurality of identification attributes;
• process the plurality of identification attributes, collectively, using a function that generates a first value, to generate an identifier for the dataset;
• generate, based on processing the plurality of identification attributes collectively and individually, a plurality of groupings;
• add a graph node, for the dataset, to a data lineage based graph representation of a plurality of datasets, wherein the graph node is associated with the identifier;
• and
• store, in a data store associated with the data lineage based graph representation of the plurality of datasets, information identifying the plurality of groupings.