n-Gram-Based Language Prediction
AssigneeAmazon Technologies, Inc.
InventorEugene Gershnik
Life of this patent
Estimating the term…
Abstract
Techniques are described for predicting the language of a text excerpt. The language prediction is accomplished by comparing n-grams of the text excerpt with n-grams of different language references. A probability is calculated for each n-gram of the text excerpt with respect to each of the language references. The calculated probabilities corresponding to a single language are then averaged to yield an overall probability corresponding to that language, and the resulting overall probabilities are compared to find the most likely language of the sample text.
Claims29
A computer-implemented method, comprising:
- under control of one or more processors configured with executable instructions,
- identifying n-gram frequencies for multiple languages;
- identifying multiple n-grams of a sample text;
- for each individual n-gram of the multiple n-grams, calculating Bayesian probabilities corresponding respectively to the multiple languages, based at least in part on the identified n-gram frequencies;
- for each individual language of the multiple languages, calculating an average of the calculated Bayesian probabilities corresponding to the individual language;
- and
- comparing the calculated averages to determine the language of the sample text.
A computer-implemented method, comprising:
- under control of one or more processors configured with executable instructions,
- identifying multiple n-grams of a sample text;
- for each individual n-gram of the multiple n-grams, calculating the probability of the individual n-gram occurring in a particular language;
- and
- averaging the calculated probabilities to indicate probability of the sample text being of the particular language.
An electronic book reader, comprising:
- a display upon which to display electronic content of different languages;
- one or more processors;
- memory containing instructions that are executable by the one or more processors to perform actions comprising:
- displaying electronic content on the display;
- accepting designation by a user of a particular word within the electronic content;
- analyzing at least a portion of the electronic content to determine its language;
- looking up the meaning of the particular word in a dictionary of the determined language;
- and
- presenting the meaning of the particular word to the user.
A computer-implemented method of evaluating probable correspondence of a sample sequence to a reference sequence, comprising:
- under control of one or more processors configured with executable instructions,
- determining frequencies of n-grams within the reference sequence;
- identifying multiple n-grams of the sample sequence;
- for each individual n-gram of the multiple n-grams, calculating the probability of the individual n-gram corresponding to the reference sequence, based at least in part on the determined frequencies of the n-grams within the reference sequence;
- and
- calculating an average of the calculated probabilities to determine the probable correspondence of the sample sequence to the reference sequence.
Dependencies are inferred from the claim text.