Systems and methods for visualizing a pattern in a dataset

A method for visualizing a pattern in a discrete attribute value dataset, the method comprising:

• at a computer system comprising a persistent memory and a non-persistent memory:
• storing the discrete attribute value dataset in persistent memory, wherein
• the discrete attribute value dataset comprises a corresponding discrete attribute value for each first entity in a plurality of first entities for each respective second entity in a plurality of second entities, and
• the discrete attribute value dataset redundantly represents the corresponding discrete attribute value for each first entity in the plurality of first entities for each respective second entity in the plurality of second entities in both a compressed sparse row format and a compressed sparse column format in which first entities for a respective second entity that have a null discrete attribute data value are discarded, and
• the discrete attribute value dataset is compressed in accordance with a blocked compression algorithm;
• clustering the discrete attribute value dataset using the discrete attribute value for each first entity in the plurality of first entities, or principal components derived therefrom, for each respective second entity in the plurality of second entities thereby assigning each respective second entity in the plurality of second entities to a corresponding cluster in a plurality of clusters, wherein
• each respective cluster in the plurality of clusters consists of a unique different subset of the second plurality of entities, and
• the clustering loads less than the entirety of the discrete attribute value dataset into the non-persistent memory at any given time during the clustering, thereby allowing the clustering of the discrete attribute value dataset having a size that exceeds storage space in the non-persistent memory allocated to the discrete attribute value dataset;
• computing, for each respective first entity in the plurality of first entities for each respective cluster in the plurality of clusters, a difference in the discrete attribute value for the respective first entity across the respective subset of second entities in the respective cluster relative to the discrete attribute value for the respective first entity across the plurality of clusters other than the respective cluster, thereby deriving a differential value for each respective first entity in the plurality of first entities for each respective cluster in the plurality of clusters;
• and
• displaying in a first panel a heat map that comprises a representation of the differential value for each respective first entity in the plurality of first entities for each cluster in the plurality of clusters thereby visualizing the pattern in the discrete attribute value dataset.

A non-transitory computer readable storage medium, wherein the non-transitory computer readable storage medium stores instructions, which when executed by a computer system, cause the computer system to perform a method for visualizing a pattern in a discrete attribute value dataset, the method comprising:

• storing the discrete attribute value dataset in persistent memory, wherein
• the discrete attribute value dataset comprises a corresponding discrete attribute value for each first entity in a plurality of first entities for each respective second entity in a plurality of second entities, and
• the discrete attribute value dataset redundantly represents the corresponding discrete attribute value for each first entity in the plurality of first entities for each respective second entity in the plurality of second entities in both a compressed sparse row format and a compressed sparse column format in which first entities for a respective second entity that have a null discrete attribute data value are discarded, and
• the discrete attribute value dataset is compressed in accordance with a blocked compression algorithm;
• clustering the discrete attribute value dataset using the discrete attribute value for each first entity in the plurality of first entities, or principal components derived therefrom, for each respective second entity in the plurality of second entities thereby assigning each respective second entity in the plurality of second entities to a corresponding cluster in a plurality of clusters, wherein
• each respective cluster in the plurality of clusters consists of a unique different subset of the second plurality of entities, and
• the clustering loads less than the entirety of the discrete attribute value dataset into the non-persistent memory at any given time during the clustering, thereby allowing the clustering of the discrete attribute value dataset having a size that exceeds storage space in the non-persistent memory allocated to the discrete attribute value dataset;
• computing, for each respective first entity in the plurality of first entities for each respective cluster in the plurality of clusters, a difference in the discrete attribute value for the respective first entity across the respective subset of second entities in the respective cluster relative to the discrete attribute value for the respective first entity across the plurality of clusters other than the respective cluster, thereby deriving a differential value for each respective first entity in the plurality of first entities for each respective cluster in the plurality of clusters;
• and
• displaying in a first panel a heat map that comprises a representation of the differential value for each respective first entity in the plurality of first entities for each cluster in the plurality of clusters thereby visualizing the pattern in the discrete attribute value dataset.