Identifying homogenous clusters
Assigneesalesforce.com, inc.
InventorGaneswara Rao DULAM
Life of this patent
Estimating the term…
Abstract
Homogeneous clusters are generated from a first plurality of documents for generation of regular expressions. Documents that share similar characteristics are clustered, and for each cluster, features are generated for use by a homogeneity model to determine a homogeneity score for the cluster. Clusters determined to be homogenous are sent to a regular expression generator.
Claims20
A computer implemented method for generating homogeneous clusters from a first plurality of documents for generation of regular expressions, the method comprising:
- clustering the first plurality of documents into a first plurality of clusters, wherein each of the first plurality of documents is included in only one of the clusters, wherein each of the clusters includes one or more of the documents and wherein the documents in each cluster share certain characteristics more closely with each other than the documents of the other clusters in the first plurality of clusters;
- for each cluster in the first plurality of clusters:
- generating a word distribution for each document in the cluster;
- assigning, using the word distribution, each word to a probability group;
- determining the percentage of words in each probability group;
- determining features for the cluster using the probability groups;
- determining a homogeneity score by applying a homogeneity model to the features for the cluster;
- and
- send those of the first plurality of clusters for which the homogeneity score exceeds a homogeneity threshold to an automatic regular expression generator.
A non-transitory machine-readable storage medium that provides instructions for generating homogeneous clusters from a first plurality of documents for generation of regular expressions that, if executed by a processor, will cause said processor to perform operations comprising:
- clustering the first plurality of documents into a first plurality of clusters, wherein each of the first plurality of documents is included in only one of the clusters, wherein each of the clusters includes one or more of the documents and wherein the documents in each cluster share certain characteristics more closely with each other than the documents of the other clusters in the first plurality of clusters;
- for each cluster in the first plurality of clusters:
- generating a word distribution for each document in the cluster;
- assigning, using the word distribution, each word to a probability group;
- determining the percentage of words in each probability group;
- determining features for the cluster using the probability groups;
- determining a homogeneity score by applying a homogeneity model to the features for the cluster;
- and
- send those of the first plurality of clusters for which the homogeneity score exceeds a homogeneity threshold to an automatic regular expression generator.
An article of manufacture for generating homogeneous clusters from a first plurality of documents for generation of regular expressions, the article comprising:
- a processor;
- and
- a memory coupled to the processor, the memory storing instructions which, when executed by the processor, cause the article to:
- cluster the first plurality of documents into a first plurality of clusters, wherein each of the first plurality of documents is included in only one of the clusters, wherein each of the clusters includes one or more of the documents and wherein the documents in each cluster share certain characteristics more closely with each other than the documents of the other clusters in the first plurality of clusters;
- for each cluster in the first plurality of clusters:
- generate a word distribution for each document in the cluster;
- assign, using the word distribution, each word to a probability group;
- determine the percentage of words in each probability group;
- determine features for the cluster using the probability groups;
- determine a homogeneity score by applying a homogeneity model to the features for the cluster;
- and
- send those of the first plurality of clusters for which the homogeneity score exceeds a homogeneity threshold to an automatic regular expression generator.
Dependencies are inferred from the claim text.