# US 10,120,860 B2 · Methods and apparatus to identify a count of n-grams appearing in a corpus > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,120,860 B2 - Document ID: US10120860B2 - Status: Granted - Kind: B2 - Application number: 15/387,059 - Filed: 2016-12-21 - Published: 2018-06-21 - Granted: 2018-11-06 - Earliest claimed: 2016-12-21 - Assignees: Intel Corporation - Applicants: Intel Corporation - Inventors: Shlomi Vaknin - Web page: https://patentgrep.com/patent/US10120860B2 ## CPC - G06F 40/284: Lexical analysis, e.g. tokenisation or collocates - G06F 40/263: Language identification - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Methods, apparatus, systems and articles of manufacture to identify a count of n-grams appearing in a corpus are disclosed herein. An example method includes identifying a token that frequently begins a suffix found in the corpus. First suffixes and second suffixes are identified within the corpus, the first suffixes begin with the token and the second suffixes do not begin with the token. A first counting algorithm is performed to identify a first count of n-grams appearing in the first suffixes. A second counting algorithm is performed to identify a second count of n-grams appearing in the second suffixes. The second counting algorithm is different from the first counting algorithm. ## Claims ### Claim 1 An apparatus to identify a count of n-grams in a corpus, the apparatus comprising: a hot-word detector to identify a token that frequently begins a suffix found in the corpus; a suffix identifier to identify first suffixes and second suffixes within the corpus, the suffix identifier to detect that the first suffixes begin with the token and that the second suffixes do not begin with the token; and an n-gram count controller to direct a cluster of nodes to perform a first counting algorithm to identify a first count of n-grams in the first suffixes, the n-gram count controller to direct the cluster of nodes to perform a second counting algorithm to identify a second count of n-grams in the second suffixes, the second counting algorithm different from the first counting algorithm, the count of n-grams used as a statistic for computational linguistics. ### Claim 2 (depends on 1) The apparatus of claim 1, wherein the cluster of nodes is to perform the second counting algorithm by distributing the second suffixes among a first number of nodes in the cluster, the first number of nodes greater than a second number of suffixes in the second suffixes. ### Claim 3 (depends on 1) The apparatus of claim 1, wherein the hot word detector is to identify the token that frequently begins the suffix based on a detected language of the corpus. ### Claim 4 (depends on 1) The apparatus of claim 1, wherein the first counting algorithm is a suffix-sigma count. ### Claim 5 (depends on 1) The apparatus of claim 1, wherein the first counting algorithm is to use a MapReduce job executed within a clustered computing environment. ### Claim 6 (depends on 1) The apparatus of claim 1, wherein each node of the cluster includes: a suffix sorter to sort the second suffixes in a reverse lexicographical order; a counter to condense the second suffixes into a list of suffixes and a number of occurrences of each suffix in the second suffixes; and an n-gram constructor to construct n-grams that begin with a prefix of each suffix in the list of suffixes, the counter to aggregate a count of the constructed n-grams to identify the second count of n-grams in the second suffixes. ### Claim 7 (depends on 6) The apparatus of claim 6, wherein the counter is to aggregate the count of the constructed n-grams based on the number of occurrences of the suffixes used to construct the corresponding n-gram. ### Claim 8 (depends on 1) The apparatus of claim 1, wherein the n-gram count controller is to combine the first count with the second count to form a total count of n-grams in the corpus. ### Claim 9 A method of identifying a count of n-grams in a corpus, the method comprising: identifying a token that frequently begins a suffix found in the corpus; identifying first suffixes and second suffixes within the corpus; detecting that the first suffixes begin with the token and that the second suffixes do not begin with the token; performing, by executing an instruction with at least one processor, a first counting algorithm to identify a first count of n-grams in the first suffixes; and performing, by executing an instruction with at the least one processor, a second counting algorithm to identify a second count of n-grams in the second suffixes, the second counting algorithm different from the first counting algorithm, the second count of n-grams used as a statistic for computational linguistics. ### Claim 10 (depends on 9) The method of claim 9, wherein the performing of the second counting algorithm includes distributing the second suffixes among a first number of nodes in a cluster, the first number of nodes greater than a second number of suffixes in the second suffixes. ### Claim 11 (depends on 9) The method of claim 9, wherein the identifying of the token that frequently begins the suffix is based on a detected language of the corpus. ### Claim 12 (depends on 9) The method of claim 9, wherein the first counting algorithm is a suffix-sigma count. ### Claim 13 (depends on 9) The method of claim 9, wherein the first counting algorithm is performed using a MapReduce job executed within a clustered computing environment. ### Claim 14 (depends on 9) The method of claim 9, wherein the second counting algorithm includes: sorting the second suffixes in a reverse lexicographical order; condensing the second suffixes into a list of suffixes and a number of occurrences of each suffix in the second suffixes; constructing n-grams that begin with a prefix of each suffix in the list of suffixes; and aggregating a count of the constructed n-grams to identify the second count of n-grams in the second suffixes. ### Claim 15 (depends on 14) The method of claim 14, wherein the aggregating of the count of the constructed n-grams is based on the number of occurrences of the suffixes used to construct the corresponding n-gram. ### Claim 16 (depends on 9) The method of claim 9, further including combining the first count with the second count to form a total count of n-grams in the corpus. ### Claim 17 A non-transitory machine-readable medium comprising instructions which, when executed, cause at least one processor to at least: identify a token that frequently begins a suffix found in the corpus; identify first suffixes and second suffixes within the corpus; detect that the first suffixes begin with the token and that the second suffixes do not begin with the token; perform a first counting algorithm to identify a first count of n-grams in the first suffixes; and perform a second counting algorithm to identify a second count of n-grams in the second suffixes, the second counting algorithm different from the first counting algorithm, the second count of n-grams used as a statistic for computational linguistics. ### Claim 18 (depends on 17) The machine-readable medium of claim 17, wherein the instructions cause the at least one processor to perform the second counting algorithm by distributing the second suffixes among a first number of nodes in a cluster, the first number of nodes greater than a second number of suffixes in the second suffixes. ### Claim 19 (depends on 17) The machine-readable medium of claim 17, instructions cause the at least one processor to identify that the token that frequently begins the suffix based on a detected language of the corpus. ### Claim 20 (depends on 17) The machine-readable medium of claim 17, wherein the first counting algorithm is a suffix-sigma count. ### Claim 21 (depends on 17) The machine-readable medium of claim 17, wherein the instructions cause the at least one processor to perform the first counting algorithm via a MapReduce job executed within a clustered computing environment. ### Claim 22 (depends on 17) The machine-readable medium of claim 17, wherein the instructions, when executed, cause the at least one processor to perform the second counting algorithm by: sorting the second suffixes in a reverse lexicographical order; condensing the second suffixes into a list of suffixes and a number of occurrences of each suffix in the second suffixes; constructing n-grams that begin with a prefix of each suffix in the list of suffixes; and aggregating a count of the constructed n-grams to identify the second count of n-grams in the second suffixes. ### Claim 23 (depends on 22) The machine-readable medium of claim 22, wherein the instructions cause the at least one processor to aggregate the count of the constructed n-grams based on the number of occurrences of the suffixes used to construct the corresponding n-gram. ### Claim 24 (depends on 17) The machine-readable medium of claim 17, wherein the instructions, when executed, further cause the at least one processor to combine the first count with the second count to form a total count of n-grams in the corpus.