Loading patent…
AssigneeThe Trustees of Columbia University in the City of New York
InventorsJudith L. Klavans, Smaranda Muresan
Estimating the term…
A system for automatically generating a dictionary from full text articles extracts <term, definition> pairs from full text articles and stores the <term, definition> pairs as dictionary entries. The system includes a computer readable corpus having a plurality of documents therein. A pattern processing module (120) and a grammar processing module (125) are provided for extracting <term, definition> pairs from the corpus and storing the <term, definition> pairs in a dictionary database (145). A routing processing module selectively routes sentences in the corpus to at least one of the pattern processing module or grammar processing module. In one embodiment, the routing module is incorporated into the pattern processing module which then selectively routes a portion of the sentences to the grammar processing module. A bootstrapping processing module (150) can be used to apply <term, definition> entries against the corpus to identify and extract additional <terms, definition> entries.
A method for automatically generating a dictionary based on a corpus of full text articles, comprising:
A method for automatically generating a dictionary based on a corpus of full text articles, comprising:
A method for automatically generating a dictionary based on a corpus of full text articles, comprising:
A method for automatically generating a dictionary based on a corpus of full text articles, comprising:
A system for automatically generating a dictionary from full text articles comprising:
A system for automatically generating a dictionary from full text articles comprising:
Computer readable media encoded with instructions to direct a computer system to generate a dictionary based on a corpus of full text articles, comprising the steps of:
Dependencies are inferred from the claim text.
Drawings are temporarily unavailable. Open the PDF at USPTO
From the US filing history. Foreign family members aren’t listed.
Lexical tools: dictionaries
Computing and data processing
Term ended 9 Nov 2024 (estimated)
The present invention relates generally to language processing and more particularly relates to a method for automatically generating dictionary entries using text analysis.
The internet has enjoyed tremendous growth over recent years. As a result, vast quantities of information are readily available to millions of users. Among the vast content available on the internet are technical papers, which are of interest to a large number of people, but may be written for a technical audience and assume a baseline understanding of terms used in the particular field. Since now, more than ever, this assumption is not necessarily true, the importance of on-line dictionaries of technical terms is of growing importance.
On-line dictionaries of technical terms have been difficult to build and are often lacking in completeness. For example, technical dictionaries such as the Online Medical Dictionary (OMD, http://www.graylab.ac.uk/omd/index.html), are often missing common terms which are assumed to be understood by those practicing in the particular field. In addition, the definitions in such dictionaries are often too technical for use by a lay person. Accordingly, it would be desirable to automatically generate on-line glossaries of technical terms that are comprehensive and generally useful to the technically oriented user as well as the lay person.
Further objects, features and advantages of the invention will become apparent from the following detailed description taken in conjunction with the accompanying figures showing illustrative embodiments of the invention, in which:
FIG. 1 is a flow diagram illustrating the overall operation of a system for generating entries for a dictionary by extracting term-definition pairs from full text articles;
FIG. 2 is a flow chart further illustrating the operation of an add linguistic markup block of FIG. 1;
Throughout the figures, the same reference numerals and characters, unless otherwise stated, are used to denote like features, elements, components or portions of the illustrated embodiments. Moreover, while the subject invention will now be described in detail with reference to the figures, it is done so in connection with the illustrative embodiments. It is intended that changes and modifications can be made to the described embodiments without departing from the true scope and spirit of the subject invention as defined by the appended claims.
FIG. 1 is a flow diagram illustrating the overall operation of a first embodiment of a system for generating entries for a dictionary by extracting <term, definition> pairs or tuples, from full text articles. While the system and methods of FIG. 1 will be described using the example of the medical field, the present invention can be applied to any body of text from which a user wishes to build a computer readable dictionary. The method of FIG. 1 is generally performed on a conventional computer system, which is operatively coupled to a source of full text articles. The articles can be stored in a local database or can be accessed over a digital communications network, such as the internet.
In step 105, the articles, in computer readable form, such as ASCII, HTML and the like, are input to the system. The articles are passed to a preprocessing algorithm, which tokenizes the input articles and formats the articles in a manner which is suitable for processing by text parser and part-of-speech tagging algorithms. The preprocessing operations of step 110 can include stripping away HTML tags, tokenizing the text and rewriting the text file as one sentence per line with each line numbered and identified to its source text. Tokenizing the text generally includes separating, by a space, each unit of a sentence (word, punctuation, number) that can be considered an independent token. Generally, hyphenated words are maintained as a single token.