Systems and Methods for Language Classification
AssigneeCrimson Hexagon, Inc.
InventorsAmac Herdagdelen, Aykut Firat, Christopher Bingham
Life of this patent
Estimating the term…
Abstract
Systems and methods are provided for classifying text based on language using one or more computer servers and storage devices. In general, the systems and methods can include a language classification module for classifying text of an input data set using the output of a training module. In an exemplary embodiment, a bootstrapping step feeds the output of the language classification module back into the training module to increase the accuracy of the language classification module. By iterating the language classification and training modules with input data having certain features, a user can tailor the language classification module for use with text having those or similar features.
Claims20
A method for classifying text according to language using one or more computer processors, comprising:
- i) statistically associating one or more features of each of a plurality of text strings of a training data set with a language in which the text string is written, using a training module that comprises a training computer processor coupled to a memory having instructions causing the training computer processor to store statistical association data in a statistical association database;
- ii) receiving an input data set having one or more text strings by a classification module that comprises a classifying computer processor coupled to a memory including instructions causing the classifying computer processor to classify one or more of the text strings according to a language in which the text string is written, tag the classified text strings with a tag that indicates the language, and output the tagged text strings;
- and
- iii) receiving by the training module the tagged text strings, statistically associating the tagged text strings with the tagged language, and updating the statistical association data in the statistical association database.
A method for classifying text according to language using one or more computer processors, comprising:
- i) accessing by a training module a training data set from a training data set database having a plurality of text strings, each text string being associated with a tag that indicates a language in which the text string is written, the training module comprising a training computer processor coupled to a memory having instructions that cause the training computer processor to access the training data;
- ii) statistically associating using the training module one or more features of each of the text strings of the training data set with its tagged language and storing statistical association data in a statistical association database;
- iii) receiving an input data set having one or more text strings by a classification module that comprises a classifying computer processor coupled to a memory including instructions that cause the classifying computer processor to classify one or more of the text strings according to a language in which the text string is written, tag the classified text strings with a tag that indicates the language, and output the tagged text strings;
- and
- iv) receiving by the training module the tagged text strings, statistically associating the tagged text strings with the tagged language, and updating the statistical association data in the statistical association database.
A computer system for classifying text according to language, comprising:
- a training module that comprises a training computer processor coupled to a memory having instructions causing the processor to receive a training data set having one or more text strings tagged with a tag that indicates a language in which the text string is written, statistically associate one or more features of each of the text strings of the training data set with its tagged language, and store statistical association data in a statistical association database;
- and
- a classification module that comprises a classifying computer processor coupled to a memory having instructions causing the processor to receive an input data set having one or more text strings, classify the one or more text strings according to a language in which the text is written, tag the classified text strings with a tag that indicates the language, and output the tagged text strings;
- wherein the training module can be configured to receive the tagged text strings output by the classifying computer processor as the training data set.
A non-transitory computer-readable storage medium containing instructions for classifying text according to language, wherein the instructions cause one or more computer processors to:
- i) access a training data set from a training data set database having a plurality of text strings, each text string being associated with a tag that indicates a language in which the text string is written;
- ii) statistically associate one or more features of each of the text strings of the training data set with its tagged language, and store statistical association data in a statistical association database;
- iii) receive an input data set having one or more text strings, classify one or more of the text strings according to a language in which the text string is written, tag the classified text strings with a tag that indicates the language, and output the tagged text strings;
- and
- iv) receive the tagged text strings, statistically associate the tagged text strings with the tagged language, and update the statistical association data in the statistical association database.
Dependencies are inferred from the claim text.