Methods and systems for using map-reduce for large-scale analysis of graph-based data

A non-volatile, non-transitory machine-readable medium containing one or more sequences of instructions for processing large-scale graph data in a distributed computing environment through a computer network coupling client computers to a server computer, the instructions configured to cause a processor to:

• execute a Markov Clustering algorithm (MCL) to find clusters of vertices of the graph data;
• organize the graph data by column by calculating a probability percentage for each column of a similarity matrix of the graph data to produce column data;
• generate a probability matrix of states of the column data;
• perform an expansion of the probability matrix by computing a power of the probability matrix using a Map-Reduce model executed in a processor-based computing device;
• and
• organize the probability matrix into a set of sub-matrices to find the least amount of data needed for the Map-Reduce model given that two lines of data in the probability matrix are required to compute a single value for the power of the probability matrix.