# US 2010/0169243 A1 · Method and system for hybrid text classification > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2010/0169243 A1 - Document ID: US20100169243A1 - Status: Published application - Kind: A1 - Application number: 12/344,480 - Filed: 2008-12-27 - Published: 2010-07-01 - Earliest claimed: 2008-12-27 - Assignees: Kibboko, Inc. - Inventors: Jiang Su; Keith Bates; Biao Wang; Bo Xu - Web page: https://patentgrep.com/patent/US20100169243A1 ## CPC - G06N 20/00: Machine learning - G06F 16/355: Creation or modification of classes or clusters - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A computer-implemented system and method for text classification is provided that applies a hybrid approach for text classification. The system and method includes a text pre-processor which prepares unclassified articles in a format which can be read by a two-stage classifier. The classifier employs a hybrid approach. A keyword-based model achieves machine-labelling of the articles. The machine-labelled articles are used to train a machine learning model. New articles can be applied against the trained model, and classified. ## Claims ### Claim 1 A computer implemented method of text classification comprising the steps of: a. receiving a set of unlabelled and a set of initially labelled documents; b. applying a keyword model to machine label the set of unlabelled documents, to produce a set of labelled (keyword) documents; c. training a machine learning model using the set of labelled (keyword) documents and the set of initially labelled documents; d. labelling a selected document using the machine learning model to produce an associated label; and e. storing the selected document and the associated label. ### Claim 2 A computer implemented method of text classification comprising the steps of: a. receiving a set of unlabelled and a set of initially labelled documents; b. applying a keyword model to machine label the set of unlabelled documents, to produce a set of labelled (keyword) documents; c. scoring the set of labelled (keyword) documents; d. if the score is less than a pre-defined threshold, then: i. training a machine learning model using the set of labelled (keyword) documents and the set of initially labelled documents; ii. labelling a selected document using the machine learning model to produce an associated label; and iii. storing the selected document and the associated label. ### Claim 3 (depends on 1) A computer implemented method of text classification according to claim 1 further comprising the step of pre-processing a set of unlabelled and a set of initially labelled documents in a vector format {w,c}. ### Claim 4 (depends on 1) A computer implemented method of text classification according to claim 1 where the machine learning model is selected from one of Na{dot over (i)}ve Bayes, Bias From Mean, Per User Average, or Per Item Average. ### Claim 5 (depends on 1) A computer implemented method of text classification according to claim 1 where the keyword model employs a Term Frequency Inverse Document Frequency weight method to generate labelled data. ### Claim 6 (depends on 2) A computer implemented method of text classification according to claim 2 further comprising the step of scoring two or more selected documents and re-training the machine learning model by receiving at least one further human-labelled document and using the at least one further human labelled document to update the machine learning model, and then further calculating the scoring of the two or more selected documents until the pre-defined threshold is reached. ### Claim 7 (depends on 1) A search or recommendation engine utilizing labels generated according to the method of claim 1. ### Claim 8 (depends on 2) A computer implemented method of text classification according to claim 2 wherein the scoring step is accomplished by using AUC and the method includes the step of receiving the pre-defined threshold. ### Claim 9 A computer implemented method of text classification comprising the steps of: a. applying a keyword model to create a label for each document in a set of documents; and b. applying a machine learning model to refine the label. ### Claim 10 (depends on 1) A computer implemented method of text classification according to claim 1 wherein the keyword model assigns a keyword based on TFIDF. ### Claim 11 (depends on 1) A computer implemented method of text classification according to claim 1 wherein the machine learning model is a supervised learning model. ### Claim 12 An apparatus for text classification comprising: a. means for storing a set of unlabelled articles; b. means for pre-processing each article; c. means for applying a keyword model to machine label each article according to a keyword; d. means for scoring the accuracy of the machine label; and e. means for applying a machine learning model to refine the machine label for each article. ### Claim 13 (depends on 1) A computer readable memory having recorded thereon statements and instructions for execution by a computer to carry out the method of claim 1. ### Claim 14 A memory for storing data for access by an application program being executed on a data processing system, comprising: a database stored in said memory, said data structure including information resident in a database used by said application program and including: a table stored in said memory serializing a set of documents and associated labels such that each label may be updated by applying a keyword model to create an initial machine label each document and a machine learning model to refine the initial machine label. ### Claim 15 (depends on 2) A computer implemented method of text classification according to claim 2 further comprising the step of pre-processing a set of unlabelled and a set of initially labelled documents in a vector format {w,c}. ### Claim 16 (depends on 2) A computer implemented method of text classification according to claim 2 where the machine learning model is selected from one of Na{dot over (i)}ve Bayes, Bias From Mean, Per User Average, or Per Item Average. ### Claim 17 (depends on 2) A computer implemented method of text classification according to claim 2 where the keyword model employs a Term Frequency Inverse Document Frequency weight method to generate labelled data. ### Claim 18 (depends on 2) A search or recommendation engine utilizing labels generated according to the method of claim 2. ### Claim 19 (depends on 2) A computer implemented method of text classification according to claim 2 wherein the keyword model assigns a keyword based on TFIDF. ### Claim 20 (depends on 9) A computer implemented method of text classification according to claim 9 wherein the keyword model assigns a keyword based on TFIDF. ### Claim 21 (depends on 2) A computer implemented method of text classification according to claim 2 wherein the machine learning model is a supervised learning model. ### Claim 22 (depends on 9) A computer implemented method of text classification according to claim 9 wherein the machine learning model is a supervised learning model.