# US 2021/0241136 A1 · Systems and methods for classifying electronic documents > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0241136 A1 - Document ID: US20210241136A1 - Status: Published application - Kind: A1 - Application number: 16/780,413 - Filed: 2020-02-03 - Published: 2021-08-05 - Earliest claimed: 2020-02-03 - Assignees: TSG Technologies, LLC - Applicants: TSG Technologies, LLC - Inventors: Caimei Lu; Ricky John Prosser; Michael Wayne Bryars - Web page: https://patentgrep.com/patent/US20210241136A1 ## CPC - G06F 16/906: Clustering; Classification - G06N 5/04: Inference or reasoning models - G06F 16/81: Indexing, e.g. XML tags; Data structures therefor; Storage structures - G06N 5/025: Extracting rules from data - G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks - G06N 20/00: Machine learning - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract A method of classifying an electronic document from an electronic media source includes generating statistical data from one or more training documents and creating a plurality of classification rules, including creating at least one topic model-based classification rule using the statistical data and creating at least one query-based classification rule using one or more user defined categories and the statistical data. The method further includes classifying the electronic document using the at least one topic model-based classification rule. Example systems for classifying an electronic document from an electronic media source are also disclosed. ## Claims ### Claim 1 A non-transitory computer readable medium having instructions stored thereon that, when executed by at least one processor, cause the at least one processor to perform operations for creating a plurality of classification rules to classify an electronic document from an electronic media source, the operations comprising: generating statistical data from one or more training documents, and creating a plurality of classification rules, including creating at least one topic model-based classification rule using the statistical data, the at least one topic model-based classification rule formatted as an XML file; evaluating the at least one topic model-based classification rule using a precision equation and a recall equation, the precision equation comprising Precision = N ⁡ ( d c + ) N ⁡ ( d c + ) + N ⁡ ( d c - ) and the recall equation comprising Recall = N ⁡ ( d c + ) N ⁡ ( d c ) , wherein N(dc+) is a number of test documents correctly classified to a category C based on the at least one topic model-based classification rule, N(dc−) is a number of test documents incorrectly classified to the category C based on the at least one topic model-based classification rule, and N(dc) denotes a number of test documents that should be classified to the category C; and creating at least one query-based classification rule using one or more user defined categories and the statistical data, the at least one query based classification rule formatted as an XML file. ### Claim 2 (depends on 1) The non-transitory computer readable medium of claim 1, the operations further comprising determining that results of the precision equation are below a defined threshold and deeming the at least one topic model-based classification rule to be unsuitable. ### Claim 3 (depends on 1) The non-transitory computer readable medium of claim 1, the operations further comprising determining that results of the recall equation are below a defined threshold and deeming the at least one topic model-based classification rule to be unsuitable. ### Claim 4 (depends on 1) The non-transitory computer readable medium of claim 1, the operations further comprising modifying the at least one topic model-based classification rule after determining that the results of the precision equation are below a defined threshold. ### Claim 5 (depends on 1) The non-transitory computer readable medium of claim 1, the operations further comprising modifying the at least one topic model-based classification rule after determining that the results of the recall equation are below a defined threshold. ### Claim 6 (depends on 1) The non-transitory computer readable medium of claim 1, the operations further comprising storing the at least one topic model-based classification rule as the XML file. ### Claim 7 (depends on 1) The non-transitory computer readable medium of claim 1, the operations further comprising repeating the evaluating the at least one topic-model based classification rule until the at least one topic-model based classification rule meets a defined threshold. ### Claim 8 A system comprising: a memory; and at least one processor to: generate statistical data from one or more training documents, and create a plurality of classification rules, including creating at least one topic model-based classification rule using the statistical data, the at least one topic model-based classification rule formatted as an XML file; evaluate the at least one topic model-based classification rule using a precision equation and a recall equation, the precision equation comprising Precision = N ⁡ ( d c + ) N ⁡ ( d c + ) + N ⁡ ( d c - ) and the recall equation comprising Recall = N ⁡ ( d c + ) N ⁡ ( d c ) , wherein N(dc+) is a number of test documents correctly classified to a category C based on the at least one topic model-based classification rule, N(dc−) is a number of test documents incorrectly classified to the category C based on the at least one topic model-based classification rule, and N(dc) denotes a number of test documents that should be classified to the category C; and create at least one query-based classification rule using one or more user defined categories and the statistical data, the at least one query-based classification rule formatted as an XML file. ### Claim 9 (depends on 8) The system of claim 8, the at least one processor further to determine that results of the precision equation are below a defined threshold and deem the at least one topic model-based classification rule to be unsuitable. ### Claim 10 (depends on 8) The system of claim 8, the at least one processor further to determine that results of the recall equation are below a defined threshold and deem the at least one topic model-based classification rule to be unsuitable. ### Claim 11 (depends on 8) The system of claim 8, the at least one processor further to modify the at least one topic model-based classification rule after determining that the results of the precision equation are below a defined threshold. ### Claim 12 (depends on 8) The system of claim 8, the at least one processor further to modify the at least one topic model-based classification rule after determining that the results of the recall equation are below a defined threshold. ### Claim 13 (depends on 8) The system of claim 8, the at least one processor further to store the at least one topic model-based classification rule as the XML file. ### Claim 14 (depends on 8) The system of claim 8, the at least one processor further to repeat the evaluating the at least one topic-model based classification rule until the at least one topic-model based classification rule meets a defined threshold. ### Claim 15 A method comprising: generating, by at least one processor, statistical data from one or more training documents, and creating a plurality of classification rules, including creating at least one topic model-based classification rule using the statistical data, the at least one topic model-based classification rule formatted as an XML file; evaluating, by the at least one processor, the at least one topic model-based classification rule using a precision equation and a recall equation, the precision equation comprising Precision = N ⁡ ( d c + ) N ⁡ ( d c + ) + N ⁡ ( d c - ) and the recall equation comprising Recall = N ⁡ ( d c + ) N ⁡ ( d c ) , wherein N(dc+) is a number of test documents correctly classified to a category C based on the at least one topic model-based classification rule, N(dc−) is a number of test documents incorrectly classified to the category C based on the at least one topic model-based classification rule, and N(dc) denotes a number of test documents that should be classified to the category C; and creating, by the at least one processor, at least one query-based classification rule using one or more user defined categories and the statistical data, the at least one query based classification rule formatted as an XML file. ### Claim 16 (depends on 15) The method of claim 15, further comprising determining that results of the precision equation are below a defined threshold and deeming the at least one topic model-based classification rule to be unsuitable. ### Claim 17 (depends on 15) The method of claim 15, further comprising determining that results of the recall equation are below a defined threshold and deeming the at least one topic model-based classification rule to be unsuitable. ### Claim 18 (depends on 15) The method of claim 15, further comprising modifying the at least one topic model-based classification rule after determining that the results of the precision equation are below a defined threshold. ### Claim 19 (depends on 15) The method of claim 15, further comprising modifying the at least one topic model-based classification rule after determining that the results of the recall equation are below a defined threshold. ### Claim 20 (depends on 15) The method of claim 15, further comprising repeating the evaluating the at least one topic-model based classification rule until the at least one topic-model based classification rule meets a defined threshold.