Back to results
US 12,609,046 B1Granted
Systems and methods for controlled language generation for language learning items
AssigneeEducational Testing Service
InventorsDebanjan Ghosh, Kevin Stowe, Mengxuan Zhao
Life of this patent
Estimating the term…
Abstract
Systems and methods are provided for generating language learning items. In embodiments, training data comprising a plurality of concept-sentence pairs received by a machine-learning based language model may be used to train the model to generate sentences based on an input. A stimulus may be received and used by the trained machine-learning based language model to generate a sentence. The sentence may be stored in a computer readable medium.
Claims18
A computer-implemented method of generating language learning items comprising:
- receiving, at a training engine, training data comprising a plurality of concept-sentence pairs from a sentence database;
- training, by a training module, a machine-learning based language model, using the training data, to generate sentences based on an input;
- evaluating, by a validation module, the trained machine-learning based language model, wherein evaluating comprises:
- receiving validation data comprising a plurality of concept-sentence pairs;
- calculating frequency counts of each concept in the validation data;
- testing the trained machine-learning based language model's ability to generate sentences from a chosen number of high frequency concepts and a chosen number of low frequency concepts based on one or more measured standards, comprising:
- sentence perplexity, concept coverage, sentence length, or lexical diversity;
- and
- retraining, by the training module, the trained machine-learning based language model if the trained machine-learning based language model does not meet a baseline value for the one or more measured standards;
- receiving, at a sentence generation engine, a stimulus;
- generating, by the sentence generation engine using the trained machine-learning based language model, a sentence based on the stimulus;
- and
- storing the sentence in a computer readable medium.
10 more elements
A system for generating language learning items comprising:
- a processing system comprising one or more data processors;
- and
- a computer-readable medium encoded with instructions for commanding the processing system to execute steps comprising:
- receiving, at a training engine, training data comprising a plurality of concept-sentence pairs from a sentence database;
- training, by a training module, a machine-learning based language model, using the training data, to generate sentences based on an input;
- evaluating, by a validation module, the trained machine-learning based language model, wherein evaluating comprises:
- receiving validation data comprising a plurality of concept-sentence pairs;
- calculating frequency counts of each concept in the validation data;
- testing the trained machine-learning based language model's ability to generate sentences from a chosen number of high frequency concepts and a chosen number of low frequency concepts based on one or more measured standards, comprising:
- sentence perplexity, concept coverage, sentence length, or lexical diversity;
- and
- retraining, by the training module, the trained machine-learning based language model if the trained machine-learning based language model does not meet a baseline value for the one or more measured standards;
- receiving, at a sentence generation engine, a stimulus;
- generating, by the sentence generation engine using the trained machine-learning based language model, a sentence based on the stimulus;
- and
- storing the sentence in a computer readable medium.
13 more elements
A non-transitory computer-readable medium encoded with instructions for commanding one or more data processors to execute steps of a method for generating language learning items comprising:
- receiving, at a training engine, training data comprising a plurality of concept-sentence pairs from a sentence database;
- training, by a training module, a machine-learning based language model, using the training data, to generate sentences based on an input;
- evaluating, by a validation module, the trained machine-learning based language model, wherein evaluating comprises:
- receiving validation data comprising a plurality of concept-sentence pairs;
- calculating frequency counts of each concept in the validation data;
- testing the trained machine-learning based language model's ability to generate sentences from a chosen number of high frequency concepts and a chosen number of low frequency concepts based on one or more measured standards, comprising:
- sentence perplexity, concept coverage, sentence length, or lexical diversity;
- and
- retraining, by the training module, the trained machine-learning based language model if the trained machine-learning based language model does not meet a baseline value for the one or more measured standards;
- receiving, at a sentence generation engine, a stimulus;
- generating, by the sentence generation engine using the trained machine-learning based language model, a sentence based on the stimulus;
- and
- storing the sentence in a computer readable medium.
10 more elements
Dependencies are inferred from the claim text.