Method and apparatus for determining associated data linkages

A method comprising:

• obtaining linkage data identifying a plurality of links for a plurality of nodes, wherein each link associates at least two of the plurality of nodes:
• a child node and a parent node having a hierarchy immediately above a hierarchy of the child node;
• partitioning the linkage data into a first number of logical partitions;
• executing, for each of the first number of logical partitions, a union find algorithm wherein each union find algorithm assigns an identifier to each node of each link based on the linkage data of each respective logical partition;
• executing a global shuffle algorithm that reassigns at least one node,
• wherein each of the at least one node has multiple node copies that were linked to different parent nodes respectively in multiple different logical partitions after executing the union find algorithm and before executing the global shuffle algorithm,
• wherein each of the at least one node is reassigned based on:
• reassigning the multiple node copies from the multiple different logical partitions to a same logical partition of the multiple different logical partitions,
• re-linking the multiple node copies, which were linked to different parent nodes in the multiple different logical partitions before executing the global shuffle algorithm, to a same parent node in the same logical partition by executing the global shuffle algorithm;
• and
• executing a path compression algorithm that compresses at least one multiple-hop link, each of which associates multiple nodes in different hierarchies respectively with multiple hops, wherein each of the at least one multiple-hop link is compressed based on:
• identifying a final parent node in a highest hierarchy among the multiple nodes,
• re-linking the remaining nodes other than the final parent node in the multiple-hop link to the final parent node directly to form multiple single-hop links.

A non-transitory computer readable medium having instructions stored thereon, wherein the instructions, when executed by at least one processor, cause a device to perform operations comprising:

• obtaining linkage data identifying a plurality of links for a plurality of nodes, wherein each link associates at least two of the plurality of nodes:
• a child node and a parent node having a hierarchy immediately above a hierarchy of the child node;
• partitioning the linkage data into a first number of logical partitions;
• executing, for each of the first number of logical partitions, a union find algorithm wherein each union find algorithm assigns an identifier to each node of each link based on the linkage data of each respective logical partition;
• executing a global shuffle algorithm that reassigns at least one node,
• wherein each of the at least one node has multiple node copies that were linked to different parent nodes respectively in multiple different logical partitions after executing the union find algorithm and before executing the global shuffle algorithm,
• wherein each of the at least one node is reassigned based on:
• reassigning the multiple node copies from the multiple different logical partitions to a same logical partition of the multiple different logical partitions,
• re-linking the multiple node copies, which were linked to different parent nodes in the multiple different logical partitions before executing the global shuffle algorithm, to a same parent node in the same logical partition by executing the global shuffle algorithm;
• and
• executing a path compression algorithm that compresses at least one multiple-hop link, each of which associates multiple nodes in different hierarchies respectively with multiple hops, wherein each of the at least one multiple-hop link is compressed based on:
• identifying a final parent node in a highest hierarchy among the multiple nodes,
• re-linking the remaining nodes other than the final parent node in the multiple-hop link to the final parent node directly to form multiple single-hop links.