Model selection in machine learning with applications to document clustering
AssigneeInternational Business Machines Corporation
InventorsByron Edward Dom, Shivakumar Vaithyanathan
Life of this patent
Estimating the term…
Abstract
A objective function based on a Bayesian statistical estimation framework is used to determine an optimal model selection by choosing both the optimal number of clusters and the optimal feature set. Heuristics can be applied to find the optimal (or at least sub-optimal) of this objective function in terms of the feature sets and the number of clusters, wherein the maximization of the objective function corresponds to the optimal model structure.
Claims18
A computer-based method for optimal clustering of data, said data retained with computer storage, said method comprising the steps:
- tokenizing a data set;
- identifying unique tokens in said tokenized data set;
- performing distributional clustering of said unique tokens to reduce noise and useful search space and identifying one or more subsets of said unique tokens;
- selecting said identified one or more subsets and calculating an objective function value for each of said selected subsets, said calculation based upon varying number of clusters over a predetermined range, and for each number of clusters, cluster said data by a clustering algorithm to form data clusters;
- choosing among said subsets the one that maximizes said objective function value, and
- wherein an optimal model for clustering said data is obtained by simultaneously determining optimal values for:
- the number of clusters, the token subset, and the data clusters, whereby said optimal values taken together, maximizes said objective function value.
A computer-based method of selecting a model for unsupervised learning comprising the steps:
- generating a list of features within a data set;
- selecting a plurality of subsets of said list of features;
- for each of said plurality of subsets, performing the following steps:
- performing clustering of data sets;
- evaluating an objective function, wherein said objective function depends on both one of said plurality of subsets and a result of clustering based on said one subset, and
- identifying an optimal subset of said plurality of subsets which maximizes said objective function.
A computer-based system for processing a collection of electronic documents, said electronic documents located local to said computer or located remotely across computer networks, said system comprising:
- a tokenizer to identify a list of features within said electronic documents;
- a selector to select a plurality of subsets of said identified features;
- a reducer to reduce said selected subsets based on a density estimation;
- a cluster mechanism to cluster documents for each of said selected subsets;
- a processing mechanism to evaluate a Bayesian function of said selected subsets, and
- wherein a model for optimizing clustering of said electronic documents is based both on the number of clusters and the subsets selected.
An article of manufacture comprising a computer useable medium having computer readable code embodied therein which selects a model for unsupervised learning, said computer readable code comprising:
- computer readable program code generating a tokenized data set;
- computer readable program code identifying all unique features or tokens in said tokenized data set;
- computer readable program code selecting a subset of said tokens;
- computer readable program code running an EM algorithm for each said selected subset;
- computer readable program code calculating an objective function for each said selected subset, and
- computer readable program code choosing among said subsets the one that optimizes the objective function.
Dependencies are inferred from the claim text.