Reducing the dissimilarity between a first multivariate data set and a second multivariate data set
AssigneeGlobal Market Insite, Inc.
InventorsMitchell Eggers, Eli Drake
Life of this patent
Estimating the term…
Abstract
A categorization data structure is described that relates to two groups of individuals. For each of two or more categories, the data structure contains (1) information identifying individuals of the first group assigned to the category, and (2) information identifying individuals of the second group assigned to the category. For each category, the information can be used to adjust the individuals of the first group assigned to the category based upon comparing the proportion of all of the individuals of the second group that are assigned to the category to the proportion of all of the individuals of the first group that are assigned to the category.
Claims36
A computer-readable storage device whose contents cause a computing system to perform a method for adjusting with respect to a plurality of attributes each having a plurality of possible values a set of survey responses each from a survey respondent and each specifying an answer to each of one or more survey questions, the method comprising:
- for each of the survey respondents, accessing a value for each of the plurality of attributes;
- for each of the plurality of members of a modeled population, accessing a value for each of the plurality of attributes;
- initializing a decision tree to contain a single node that is both a root node and a leaf node;
- completing the initialized decision tree by adding interior nodes to the decision tree that each reduce the total dissimilarity measure between the reference and subject data sets by specifying a partitioning of the possible values of one of the plurality of attributes, wherein the dissimilarity measure of each leaf node is determined based on the level of dissimilarity of the attribute values of the survey respondents whose attribute values satisfy any partitionings specified by interior nodes on the path from the root node to the leaf node with the attribute values of the members of the modeled population whose attribute values satisfy any partitionings specified by interior nodes on the path from the root node to the leaf node;
- for each of the leaf nodes in the completed decision tree:
- identifying any members of the modeled population whose attribute values satisfy the partitionings specified by the path within the tree from its root node to the leaf node,
- identifying any survey respondents whose attribute values satisfy the partitionings specified by the interior nodes on the path from the root node to the leaf node, and
- determining a ratio of the number or proportion of members of the modeled population identified for the leaf node to a number of survey respondents identified for the leaf node;
- for each of the survey responses, adjusting each answer to a survey question specified by the survey response by a factor based upon the ratio determined for the leaf node for which the survey respondent from whom the survey response was received was identified.
A method in a computing system for generating a binary partitioning decision tree with respect to a pair of groups each comprising a plurality of entities, each of the entities having, for each of a plurality of variables, a value among a set of possible values, comprising:
- initializing a decision tree to contain a single node that is both a root node and a leaf node;
- and
- completing the initialized decision tree by repeatedly, until one of one or more limits on the extent of the decision tree is reached:
- for each of the leaf nodes in the decision tree:
- identifying any entities of the first group whose variable values satisfy any partitionings specified by interior nodes on the path within the tree from its root node to the leaf node;
- identifying any entities of the second group whose variable values satisfy any partitionings specified by interior nodes on the path from the root node to the leaf node;
- and
- attributing a weighting to the leaf node that is based on a measure of the dissimilarity of the variable values of the entities of the first group identified for the leaf node with the variable values of the entities of the second group identified for the leaf node relative to a measure of the dissimilarity of the variable values of all of the entities of the first group with the variable values of all of the entities of the second group,
- stochastically selecting one of the leaf nodes of the tree in accordance with the weightings attributed to the leaf nodes,
- attributing a score to each possible partitioning of the possible values of each of the plurality of variables that is based on, if a new node specifying the possible partitioning and having first and second child nodes was substituted for the selected leaf node, (1) a dissimilarity measure of the variable values of the entities of the first group whose variable values satisfy any partitionings specified by interior nodes on the path from the root node to the first child node with the variable values of the entities of the second group whose variable values satisfy any partitionings specified by interior nodes on the path from the root node to the first child node and (2) a dissimilarity measure of the variable values of the entities of the first group whose variable values satisfy any partitionings specified by interior nodes on the path from the root node to the second child node with the variable values of the entities of the second group whose variable values satisfy any partitionings specified by interior nodes on the path from the root node to the second child node,
- stochastically selecting one possible partitioning of the possible values of one of the variables in accordance with the scores attributed to the partitionings, and
- replacing the selected leaf node with a new interior node, the interior node having as children two new leaf nodes and branching among its children based upon the selected partitioning.
A method in a computing system for adjusting survey responses from a subject group of individuals with reference to a reference group of individuals, comprising:
- constructing a decision tree that classifies individuals into categories based on each individual's values for a plurality of attributes;
- for each category, using the ratio of the number of individuals of the reference group classified into the category to the number of individuals of the subject group classified into the category to adjust both attributes and survey responses from the individuals of the subject group classified into the category;
- and
- determining a dissimilarity measure between the adjusted attributes of the individuals of the subject group and the attributes of the individuals of the reference group that is smaller than a dissimilarity measure between the pre-adjusted attributes of the subject group with the attributes of the reference group.
A computer-readable storage device whose contents are capable of causing a computing system to perform a method for adjusting data sets among a first data set of individuals and a second data set of individuals, the method comprising:
- constructing a decision tree that classifies individuals into categories based on each individual's values for a plurality of attributes;
- for each category, using the ratio of the number of individuals of the first data set classified into the category to the number of individuals of the second data set classified into the category to adjust one or more individuals of at least one of the first data set and the second data set classified into the category;
- and
- determining a total dissimilarity measure between the attributes of the individuals of the first data set after any adjustment and the attributes of the individuals of the second data set after any adjustment that is smaller than a dissimilarity measure between the attribute values of the individuals of the first data set before any adjustment and the attribute values of the individuals of the second data set before any adjustment.
A computer-readable storage device whose contents are capable of causing a computing system to perform a method for combining a plurality of subject groups with respect to a reference group, each group comprising a plurality of entities, each of the entities having, for each of a plurality of variables, a value among a set of possible values, the method comprising:
- experimentally selecting a plurality of sets of weights for each of the subject groups;
- for each selected set of weights:
- applying the weights to the variable values of each of the subject groups;
- combining the entities of the subject groups having the weighted variable values into a combined subject group;
- determining a dissimilarity measure for the set that measures how dissimilar the weighted variable values of the entities of the combined subject group are from the variable values of the entities of the reference group;
- and
- storing the combined subject group generated using the set of weights having the smallest dissimilarity measure.
A method in a computing system for dividing a pool of entities into two groups that are balanced on a set of attributes, each of the entities of the pool having a value for each attribute of the set, comprising:
- repeating:
- establishing an allocation of the pool that allocates each of the entities to one of the two groups, and
- determining a measure of the dissimilarity of the attribute values of the individuals of the two groups that incorporates both (1) a marginal distribution of each attribute among individuals of each of the two groups, and (2) each attribute's interrelation with every other variable among individuals of each of the two groups until the last-determined measure is within a preselected threshold;
- and
- selecting the last established location;
- and
- for a distinguished established allocation, constructing a decision tree that classifies entities into categories based on each entity's values for the attributes of the set, such that, for each of the categories, a measure of the dissimilarity of the attribute values of the entities of the two groups classified into the category is smaller than a measure of the dissimilarity of the attribute values of all of the entities of the two groups, and wherein the establishment of the allocation following the allocation involves exchanging between the groups of the selected allocation one or more pairs of entities classified into the category having the highest measure of dissimilarity.
Dependencies are inferred from the claim text.