Systems and methods for constructed response scoring using metaphor detection

A computer-implemented method of scoring a text based on at least predicted figurative word usage in the constructed texts, the method comprising:

• accessing a text to be evaluated with a processing system;
• identifying content words in the text with the processing system;
• extracting one or more features from each of the content words with the processing system;
• predicting, with a processing system, whether each of the content words is being used figuratively in the text, the predicting being based on a prediction model and the extracted one or more features;
• and
• generating an evaluation score with the processing system for the text based on the predictions;
• wherein the one or more features include at least one of a topic model feature, a unigram feature, a part-of-speech feature, a concreteness feature, a concreteness difference feature, a literal context feature, a non-literal context feature, and an off-topic feature.

A system for generating a model for predicting whether a content word is being used figuratively, comprising:

• a processing system;
• and
• a memory;
• wherein the processing system is configured to execute steps comprising:
• accessing a plurality of training texts, each training text including content words, and each content word having a corresponding annotation indicating whether the content word is being used figuratively in the associated training text;
• accessing a plurality of topic models generated from a first corpus;
• for each content word in the plurality of training texts, assigning a plurality of topic-model feature scores to the content word, each topic-model feature score being associated with one of the plurality of topic models and determined based on:
• determining a first probability of the content word being used in a topic represented by the topic model;
• determining a second probability of the content word being used in a second corpus;
• and
• computing the topic-model feature score using the first probability and the second probability;
• and
• generating a prediction model for predicting whether a content word is being used figuratively, the generating being based on at least one of the plurality of topic-model feature scores and the annotations.

A system for scoring a text based on at least predicted figurative word usage in the constructed texts, comprising:

• a processing system;
• and
• a memory;
• wherein the processing system is configured to execute steps comprising:
• accessing a text to be evaluated;
• identifying content words in the text;
• extracting one or more features from each of the content words;
• predicting whether each of the content words is being used figuratively in the text, the predicting being based on a prediction model and the extracted one or more features;
• and
• generating an evaluation score for the text based on the predictions;
• wherein the one or more features include at least one of a topic model feature, a unigram feature, a part-of-speech feature, a concreteness feature, a concreteness difference feature, a literal context feature, a non-literal context feature, and an off-topic feature.

A non-transitory computer-readable medium for generating a model for predicting whether a content word is being used figuratively, comprising instructions which when executed cause a processing system to carry out steps comprising:

• accessing a plurality of training texts, each training text including content words, and each content word having a corresponding annotation indicating whether the content word is being used figuratively in the associated training text;
• accessing a plurality of topic models generated from a first corpus;
• for each content word in the plurality of training texts, assigning a plurality of topic-model feature scores to the content word, each topic-model feature score being associated with one of the plurality of topic models and determined based on:
• determining a first probability of the content word being used in a topic represented by the topic model;
• determining a second probability of the content word being used in a second corpus;
• and
• computing the topic-model feature score using the first probability and the second probability;
• and
• generating a prediction model for predicting whether a content word is being used figuratively, the generating being based on at least one of the plurality of topic-model feature scores and the annotations.

A non-transitory computer-readable medium for scoring a text based on at least predicted figurative word usage in the constructed texts, comprising instructions which when executed cause a processing system to carry out steps comprising:

• accessing a text to be evaluated;
• identifying content words in the text;
• extracting one or more features from each of the content words;
• predicting whether each of the content words is being used figuratively in the text, the predicting being based on a prediction model and the extracted one or more features;
• and
• generating an evaluation score for the text based on the predictions;
• wherein the one or more features include at least one of a topic model feature, a unigram feature, a part-of-speech feature, a concreteness feature, a concreteness difference feature, a literal context feature, a non-literal context feature, and an off-topic feature.

A computer-implemented method of scoring a text based on at least predicted figurative word usage in the constructed texts, the method comprising:

• accessing a text to be evaluated with a processing system;
• identifying content words in the text with the processing system;
• extracting one or more features from each of the content words with the processing system, wherein extracting one or more features includes determining whether a particular content word belongs in a particular word group, wherein the particular word group is associated with a figurative usage likelihood;
• predicting, with a processing system, whether each of the content words is being used figuratively in the text, the predicting being based on a prediction model and the extracted one or more features;
• and
• generating an evaluation score with the processing system for the text based on the predictions.