Content filtering for electronic documents generated in multiple foreign languages
AssigneeInternational Business Machines Corporation
InventorJOAQUIN M. MARQUES
Life of this patent
Estimating the term…
Abstract
A system for collecting and categorizing metadata about content provided via the internet or intranet, regardless of the language of generation of the content. The content of each document is assigned token IDs, which token IDs are the same for any given topic irrespective of the language in which the document is written. Filtering of single language documents will generate a single output; whereas, multilingual documents will be divided into language segments with each segment being filtered by the appropriate language filter.
Claims13
A method for categorizing documents generated in one or more languages comprising the steps of:
- providing topic categories representing subject matter from documents;
- assigning topic token IDs to said topic categories regardless of language of generation;
- replacing document content with at least one document token ID for each of said topic categories;
- and
- matching topic token IDs to said at least one document token ID.
A system for categorizing documents according to topic categories, said documents having been generated in one or more languages comprising:
- means for identifying languages in which said documents were generated;
- means for marking said documents where said identified languages appear;
- means for assigning topic token IDs to said topic categories;
- means for replacing document content with at least one document token ID for each of said topic categories;
- and
- a plurality of document filter means, one for each of said one or more languages, each of said plurality of document filter means being adapted to recognize said document token IDs and match said topic token IDs to said at least one document token ID.
Dependencies are inferred from the claim text.