Cluster processing and ranking methods including methods applicable to clusters developed through density based merging

A method of using an information system to cluster data items in a set of multi-dimensional data items corresponding to flow cytometry data of individuals, wherein a data item is associated with one or more semi-continuous values, the method comprising:

• obtaining or accessing the set of data items corresponding to flow cytometry data of individuals with each data item corresponding to an observation of an individual and each observation having d dimensions, where each dimension represents a semi-continuous value relative to the observation, a semi-continuous value being a value of a variable that can take the value zero or any value above a positive threshold;
• creating a reduction data item set, each reduction data item in the reduction data item set associated with one or more quantized values correlated with the one or more semi-continuous values for one or more data items;
• assigning each data item to a reduction data item according to an assignment rule;
• calculating weights for the reduction data items using one or more data items assigned to the reduction data item according to a weighting rule;
• determining an estimated density surface based on the weights assigned to each reduction data item;
• determining for each of a plurality of reduction data items if the reduction data item should be associated with another reduction data item according to an association rule based on the estimated density surface, and if so, creating a directional association of the reduction data item with the other reduction data item;
• and
• following directional associations between reduction data items and determining a terminal state for each of one or more pointer paths formed by the directional associations between reduction data items by identifying a last reduction data item in the pointer path that does not have a pointer to another reduction data item originating from it as a cluster root or as background noise based on a comparison between a density estimate for the last reduction data item and a density estimate threshold;
• identifying one or more clusters of the reduction data items using one or more directional associations and/or one or more of the weights, thereby identifying clusters of the data items based on the identification of the associated reduction data items.

An apparatus for creating groupings of data items in a set of multi-dimensional data items corresponding to flow cytometry data of individuals, the apparatus comprising:

• storage configured to store the set of multi-dimensional data items corresponding to flow cytometry data of individuals with each data item corresponding to an observation of an individual and each observation having d dimensions where each dimension represents a value relative to the observation;
• and
• one or more processors configured to execute instructions, that, when executed by the processor, cause the apparatus to:
• define a plurality of lattice points in a data space able to contain at least some of the set of data items, the lattice points spaced according to a lattice rule and used as data reduction points in the data space;
• assign each data item from the set of data items to a point on a lattice;
• assign a weight to each lattice point based on one or more data items near the lattice point;
• determine an estimated density surface based on the weights assigned to each lattice point;
• determine, for each of the lattice points, if the lattice point should be associated with one of its surrounding lattice points based on the estimated density surface, and, if so, associate the lattice point with the one of its surrounding lattice points by creating a pointer from the individual lattice point to the surrounding lattice point;
• follow directional associations between lattice points and determine a terminal state for each of one or more pointer paths formed by the directional associations between lattice points by identifying a last lattice point in the pointer path that does not have a pointer to another lattice point originating from it as a cluster root or as background noise based on a comparison between a density estimate for the last lattice point and a density estimate threshold;
• assign each lattice point to a cluster or to background noise based on associations between the lattice points and a values of the estimated density surface at lattice points;
• and
• assign each data item contained in the data space to a cluster or to background noise based on the assignment of the corresponding lattice point.

A computer implemented method for guiding clustering strategy in identifying clusters of data items in a set of multi-dimensional data items corresponding to flow cytometry data of individuals, the method comprising:

• obtaining or accessing the set of data items corresponding to flow cytometry data of individuals with each data item corresponding to an observation of an individual and each observation having d dimensions where each dimension represents a value relative to the observation and where d is greater than two;
• clustering the data items based on a first pair of the d dimensions, the clustering the data items based on the first pair of the d dimensions comprising:
• defining a plurality of lattice points in a data space able to contain at least some of the set of data items;
• calculating weights for the lattice points;
• determining an estimated density surface relating a weight at a lattice point to weights at nearby lattice points using a density function;
• creating, for each lattice point in at least a subset of the lattice points, a directional association with at least one other lattice point using an association rule based, at least in part, the estimated density surface at the lattice point;
• following directional associations between lattice points and determining a terminal state for each of one or more pointer paths formed by the directional associations between lattice points by identifying a last lattice point in the pointer path that does not have a pointer to another lattice point originating from it as a cluster root or as background noise based on a comparison between a density estimate for the last lattice point and a density estimate threshold;
• assigning each lattice point in the pointer path to a cluster corresponding to the cluster root or to background noise based on the determined terminal state of the pointer path;
• clustering the data items based on a second pair of the d dimensions, the second pair not identical to the first pair;
• for each clustering of the data items based on a dimension pair, calculating an associated dimension pair entropy-based ranking score;
• displaying to a user results of an automated dimension pair entropy-based ranking based on the calculated dimension pair entropy-based ranking score for each dimension pair used for clustering;
• and
• receiving a selection of a dimension pair from the user and implementing the user's selection of the dimension pair to be employed for clustering.