Method and system for confidence-weighted learning of factored discriminative language models

A method, for building a language model for a translation system, comprising:

• providing a first relative ranking of first and second translations in a target language of a same source string in a source language;
• determining a second relative ranking of the first and second translations using weights of a language model, the language model including a weight for each of a set of n-gram features, the determining of the second ranking comprising:
• determining a first set of counts, each count representing a number of times a respective n-gram feature occurs in the first translation;
• determining a second set of counts, each count representing a number of times the respective n-gram feature occurs in the second translation;
• computing a set of differences between the counts in the first and second sets of counts;
• and
• computing a function of the set of weights and the set of differences;
• comparing the first and second relative rankings to determine whether they are in agreement;
• and
• where, based on the comparison, the rankings are not in agreement, updating one or more of the n-gram feature weights in the language model as a function of a measure of confidence in the weight, the measure of confidence being a function of previous observations of the n-gram feature in the method, comprising:
• determining a learning rate based on a current weight of each n-gram feature identified in the first and second translations having a non-zero computed difference, the computed difference for each n-gram feature, and the confidence measure for each n-gram feature weight;
• and
• updating the n-gram feature weight as a function of the learning rate and the computed difference for the n-gram feature.

A method for building a language model for a translation system, comprising:

• providing a first relative ranking of first and second translations in a target language of a same source string in a source language;
• determining a second relative ranking of the first and second translations using weights of a language model, the language model including a weight for each of a set of n-gram features;
• comparing the first and second relative rankings to determine whether they are in agreement, the determination of whether the first and second rankings are in agreement comprises determining whether:
• wtTΔ≦a threshold value,
• where wt is a vector of the n-gram feature weights, which includes a weight for each n-gram feature in the set of n-gram features, at time t;
• Δ is a vector of differences, which includes one difference per n-gram feature in the set of n-gram features for the first and second translations;
• and
• T indicates a transpose operation;
• and
• where, based on the comparison, the rankings are not in agreement, updating one or more of the weights in the language model as a function of a measure of confidence in the weight, the measure of confidence being a function of previous observations of the n-gram feature in the method.

A system for building a language model for a translation system comprising:

• memory which stores:
• a language model comprising a set of weights including a weight for each of a set of n-gram features which have been identified in a corpus of sentences in a target language;
• a first ranking component which provides a first relative ranking for first and second machine translations in the target language of a same source string in a source language, based on a scoring metric;
• a second ranking component which provides a second relative ranking for the first and second translations based on counts of n-gram features in each of the first and second machine translations and the weights of those n-gram features in the language model;
• a comparison component which compares the first and second relative rankings and, where the first and second relative rankings are not in agreement, updates at least one of the weights in the language model for an n-gram feature identified in at least one of the first and second translations, the updating being based on a measure of confidence in the at least one weight, the confidence being a function of previous observations of the n-gram feature in the method;
• and
• a processor which implements the first ranking component, second ranking component, and comparison component.

A method for updating weights in a language model comprising:

• a) providing a language model in memory comprising a vector of weights, one weight for each of a set of n-gram features, and a confidence measure being associated with each weight;
• b) with a computer processor, for first and second translations of a same source sentence, the first translation of the source sentence being more positively ranked than the second translation of the source sentence, based on a translation scoring metric:
• computing an n-gram feature vector for each of the first and second translations of the source sentence, each n-gram feature vector representing a number of occurrences in the respective translation, of n-gram features having a weight in the language model;
• computing a difference vector by subtracting the n-gram feature vector of the second translation from the n-gram feature vector of the first translation;
• computing a function of the difference vector and the vector of weights;
• and
• when the computed function has a value which does not at least equal a predetermined threshold:
• updating the vector weights based on the difference vector and confidence measures, and
• updating the confidence measures to reflect observations of the n-gram features in the first and second translations;
• and
• c) repeating b) at least once with at least one of:
• a different pair of first and second translations of the same source sentence;
• and
• first and second translations of a new source sentence.