Method and apparatus for training a text classifier

#The present invention relates generally to computerized text classification. More particularly, the present invention relates to the combined supervised and unsupervised learning of text classifiers.

#The amount and variety of data stored on-line is growing at a rapid rate. This is particularly true for natural language text in its many forms (news articles, memos, electronic mail, repair reports, etc.). While there are many potential benefits of computer access to this data, they cannot be realized unless documents useful under particular circumstances can be distinguished from ones which are not useful.

#An important technique in on-line text processing is text classification, which is the sorting of documents into meaningful groups. A variety of text classification systems are currently in use. Text retrieval systems attempt to separate documents from a text database into two groups: those which are relevant to a user query and those which are not. Text routing systems, or filtering systems, direct documents from an incoming stream of documents to relevant users. Text categorization systems sort documents into two or more designated classes. Text classification can be applied to documents which are purely textual, as well as documents which contain both text and other forms of data.

#These and other advantages of the invention will be apparent to those of ordinary skill in the art by reference to the following detailed description and accompanying drawings.

#FIG. 1 shows a schematic of the components of a computer system which can be configured to implement the present invention.

#FIG. 2 is a block diagram of an overview of the components of the present invention.

#As used herein, the term computer includes any device or machine capable of accepting data, applying prescribed processes to the data, and supplying the results of the processes.

#The functions of the present invention are preferably performed by a programmed digital computer of the type which is well know in the art, an example of which is shown in FIG. 1. FIG. 1 shows a computer system 100 which comprises a display monitor 102, a textual input device such as a computer keyboard 104, a graphical input device such as a mouse 106, a computer processor 108, a memory unit 110, and a non-volatile storage device such as a disk drive 120. The memory unit 110 includes a storage area 112 for the storage of, for example, computer program code, and a storage area 114 for the storage of data. The computer processor 108 is connected to the display monitor 102, the memory unit 110, the non-volatile storage device 120, the keyboard 104, and the mouse 106. The external storage device 120 may be used for the storage of data and computer program code. The computer processor 108 executes the computer program code which is stored in the memory unit 110 in storage area 112. During execution, the processor may access data in storage space 114 in the memory unit 110, and may access data stored in the non-volatile storage device 120. The computer system 100 may suitably be any one of the types which are well known in the art such as a mainframe computer, a minicomputer, a workstation, or a personal computer.

#FIGS. 2 and 3 are block diagrams of components of the present invention, and will be discussed in further detail below. It will be understood by those skilled in the art that the components of the present invention as shown in FIGS. 2 and 3 may advantageously be implemented using an appropriately programmed digital computer as shown in FIG. 1.