Semi-supervised part-of-speech tagging
AssigneeMicrosoft Corporation
InventorsKristina Nikolova Toutanova, Mark Edward Johnson
Life of this patent
Estimating the term…
Abstract
A word is selected from a received text and features are identified from the word. The features are applied to a model to identify probabilities for sets of part-of-speech tags. The probabilities for the sets of part-of-speech tags are used to weight scores for possible part-of-speech tags for the selected word to form weighted scores. The weighted scores are used to select a part-of-speech tag for the word and the selected part of speech tag is stored or output. The scores for the possible part-of-speech tags are based on variational approximation parameters trained from a sparse prior over probability distributions describing the probability of a part-of-speech tag given a word.
Claims20
A method comprising:
- receiving a text comprising a sequence of words;
- selecting a word from the text;
- identifying features of the selected word, the features comprising a suffix of the selected word;
- applying the features of the selected word to a model to identify probabilities for sets of part-of-speech tags, at least one set of part-of-speech tags comprising at least two part-of-speech tags, each part-of-speech tag representing a part-of-speech;
- using the probabilities for sets of part-of-speech tags to weight scores for possible part-of-speech tags for the selected word to form weighted scores;
- using the weighted scores to select a part-of-speech tag for the selected word;
- and
- storing the selected part-of-speech tag for the selected word.
The method of 6 wherein the model is trained by forming partial counts of part-of-speech tags based on a probability of a part-of-speech tag given a set of features.
A computer-readable storage medium having encoded thereon computer-executable instructions causing a processor to execute steps comprising:
- receiving a text comprising a sequence of words;
- training variational approximation parameters based on a sparse prior distribution of probability distributions that describe the probability of a part-of-speech tag given a word and based on the sequence of words, the variational approximation parameters comprising a separate variational approximation parameter for each occurrence of each word in the sequence of words, each separate variational approximation parameter describing a distribution for a tag given a word;
- selecting a part-of-speech tag for a word that maximizes a value computed from the distributions formed by the variational approximation parameters;
- and
- outputting the selected part-of-speech tag as the part-of-speech tag for the word.
A method comprising:
- receiving a text;
- selecting a first word in the text;
- retrieving an entry for the first word from a dictionary stored on a computer-readable storage medium, the entry indicating a set of part-of-speech tags associated with the first word;
- using the set of part-of-speech tags from the entry to identify a part-of-speech tag for the first word;
- storing the part-of-speech tag for the first word on a computer-readable storage medium;
- selecting a second word in the text;
- determining that the dictionary does not have an entry for the second word;
- selecting a part-of-speech tag for the second word based in part on probabilities of sets of part-of-speech tags given features of the second word;
- and
- storing the part-of-speech tag for the second word on a computer-readable storage medium.
Dependencies are inferred from the claim text.