Unsupervised information extraction dictionary creation
AssigneeInternational Business Machines Corporation
InventorsSheng Hua Bao, Su Yan
Life of this patent
Estimating the term…
Abstract
A data handling system enables the unsupervised creation of an information extraction dictionary by expanding upon a word or phrase included within an expansion query. Prior to receiving the expansion query, the data handling system performs an unsupervised learning of an information corpus which includes text to assign a corpus vector to each word and phrase of the text. After the expansion query, the data handling system compares the expansion query to the corpus vectors. The data handling system ranks the corpus vectors by similarity to the expansion query and provides a ranked list of words or phrases associated with the ranked corpus vectors. The ranked list may be subsequently utilized as the information extraction dictionary.
Claims20
A method of performing an unsupervised learning of text within an information corpus to generate a vector representation of every word or phrase of the text of the information corpus, the method comprising:
- prior to a host device receiving an expansion query from a client device, assigning, with the host device, a corpus vector to each word and phrase within an information corpus stored within a data source local to the host system;
- forming, with the host device, a plurality of clusters, each cluster comprising a plurality of similar corpus vectors;
- and
- indicating, with the host device, a particular corpus vector within each cluster as being a representative corpus vector of the cluster in which the particular corpus vector resides.
A computer program product for performing an unsupervised learning of text within an information corpus to generate a vector representation of every word or phrase of the text of the information corpus, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions readable by a host device to cause the host device to:
- prior to the host device receiving an expansion query from a client device, assign a corpus vector to each word and phrase within an information corpus stored within a data source local to the host system;
- form a plurality of clusters, each cluster comprising a plurality of similar corpus vectors;
- and
- indicate a particular corpus vector within each cluster as being a representative corpus vector of the cluster in which the particular corpus vector resides.
A computer for performing an unsupervised learning of text to generate a vector representation of every word or phrase of the text, the computer comprising:
- a processor;
- an information corpus stored within a data source communicatively coupled to the processor;
- and
- a memory communicatively coupled to the processor, wherein the memory is encoded with instructions, wherein the instructions when executed by the processor cause the processor to:
- prior to the processor receiving an expansion query from a client device, assign a corpus vector to each word and phrase of text within the information corpus;
- form a plurality of clusters, each cluster comprising a plurality of similar corpus vectors;
- and
- indicate a particular corpus vector within each cluster as being a representative corpus vector of the cluster in which the particular corpus vector resides.
Dependencies are inferred from the claim text.