Back to results
US 7,127,208 B2Granted
Automated annotation
AssigneeEducational Testing Service
InventorsJill Burstein, Daniel Marcu
Life of this patent
Estimating the term…
Abstract
To automatically annotate an essay, a sentence of the essay is identified and a feature associated with the sentence is determined. In addition, a probability of the sentence being a discourse element is determined by mapping the feature to a model. The model having been generated by a machine learning application based on at least one annotated essay. Furthermore, the essay is annotated based on the probability.
Claims54
A method of annotating an essay, said method comprising:
- loading an essay into a memory of an electronic device;
- identifying a sentence of the essay;
- determining a feature associated with said sentence;
- determining a probability of said sentence being a discourse element by mapping said feature to a model, said model having been generated by a machine learning application based on at least one annotated essay;
- and
- annotating said essay based on said probability,
- wherein said feature comprises a positional feature.
4 more elements
A method of annotating an essay, said method comprising:
- identifying a sentence of an essay;
- generating a flat file for said essay, said flat file including an entry for said sentence;
- determining a positional feature associated with said sentence;
- modifying said entry to include data associated wit said positional feature;
- identifying a lexical feature associated with said sentence;
- modifying said entry to include data associated with said lexical feature;
- identifying a rhetorical feature associated with said sentence;
- modifying said entry to include data associated with said rhetorical feature;
- determining a probability of said sentence being a discourse element by mapping said flat file to a model, said model having been generated by a machine learning application based on at least one annotated essay;
- and
- annotating said essay based on said probability.
8 more elements
A method of annotating an essay, said method comprising:
- receiving an essay;
- identifying a sentence of said essay;
- generating a flat file for said essay, said flat file including an entry for said sentence;
- determining a positional feature associated with said sentence, wherein said positional feature comprises at least one of:
- a sentence position, said sentence position being associated with a position of said sentence within said essay;
- a relative sentence position, said relative sentence position being associated with a relative position of said sentence within said essay;
- a paragraph position, said paragraph position being associated with a position of said sentence within a paragraph of said essay;
- and
- a relative paragraph position, said relative paragraph position being associated with a relative position of said paragraph within said essay;
- modifying said entry to include data associated with said positional feature;
- identifying a lexical feature associated with said sentence, wherein said lexical feature comprises at least one of:
- a category-specific cue, said category-specific cue being typically associated with a discourse element;
- a general vocabulary cue, said general vocabulary cue being typically associated with a discourse structure;
- and
- a key term, said key term being typically associated with a discourse relationship;
- modifying said entry to include data associated with said lexical feature;
- identifying a punctuation associated with said sentence;
- modifying said entry to include data associated with said punctuation;
- generating a rhetorical structure tree based on said flat file;
- identifying a rhetorical feature based on said rhetorical structure tree, wherein said rhetorical feature comprises at least one of:
- a discourse structure, said discourse structure being typically associated with an elementary discourse unit;
- a rhetorical relation, said rhetorical relation describing a manner of association between a plurality of said discourse structures;
- and
- a status, said status comprising:
- a nucleus, said nucleus being associated with a relatively more important one of said plurality of discourse structures;
- and
- a satellite, said satellite being associated with a relatively less important one of said plurality of discourse structures;
- modifying said entry to include data associated with said rhetorical feature;
- determining a probability of said sentence being a discourse element by mapping said flat file to a model, said model having been generated by a machine learning application based on at least one annotated essay;
- and
- annotating said essay based on said probability.
28 more elements
A computer readable medium on which is embedded computer software, said software comprising executable code for performing a method comprising:
- identifying a sentence of an essay;
- determining a feature associated with said sentence;
- determining a probability of said sentence being a discourse element by mapping said feature to a model, said model having been generated by a machine learning application based on at least one annotated essay;
- and
- annotating said essay based on said probability,
- wherein said feature comprises a positional feature.
3 more elements
An automatic essay annotator comprising:
- means for identifying a sentence of an essay;
- means for determining a feature associated with said sentence;
- means for determining a probability of said sentence being a discourse element, said means for determining said probability being configured to map said feature to a model, said model having been generated by a machine learning application based on at least one annotated essay and said discourse element being at least one of:
- title;
- background;
- thesis statement;
- main points;
- support;
- and conclusion;
- and
- means for annotating said essay based on said probability,
- wherein said means for determining said feature further comprises means for determining a positional feature.
9 more elements
An automatic essay annotator comprising:
- a feature extractor, said feature extractor comprising:
- a position identifier configured to determine a positional feature associated with a sentence of said essay, said position identifier further configured to generate a flat file, said flat file including an entry for said sentence, said entry including data associated with said positional feature;
- a lexical item identifier configured to identify a lexical feature associated with said sentence, said lexical item identifier further configured to modify said entry to include data associated with said lexical feature;
- and
- a rhetorical relation identifier configured to identify a rhetorical feature, said rhetorical relation identifier being further configured to modify said entry to include data associated with said rhetorical feature;
- and
- a discourse analysis modeler configured to determine a probability of said sentence being a discourse element, said discourse analysis modeler being configured to determine said probability by mapping said flat file to a model, said model having been generated by a machine learning application based on at least one annotated essay, said discourse analysis modeler being further configured to annotate said essay based on said probability.
4 more elements
A method of annotating an essay, said method comprising:
- loading an essay into a memory of an electronic device;
- identifying a sentence of the essay;
- determining a feature associated with said sentence;
- determining a probability of said sentence being a discourse element by mapping said feature to a model, said model having been generated by a machine learning application based on at least one annotated essay;
- and
- annotating said essay based on said probability,
- wherein said feature comprises a lexical feature.
4 more elements
A method of annotating an essay, said method comprising:
- loading an essay into a memory of an electronic device;
- identifying a sentence of the essay;
- determining a feature associated with said sentence;
- determining a probability of said sentence being a discourse element by mapping said feature to a model, said model having been generated by a machine learning application based on at least one annotated essay;
- and
- annotating said essay based on said probability,
- wherein said feature comprises a rhetorical feature.
4 more elements
A method of annotating an essay, said method comprising:
- loading an essay into a memory of an electronic device;
- identifying a sentence of the essay;
- determining a feature associated with said sentence;
- determining a probability of said sentence being a discourse element by mapping said feature to a model, said model having been generated by a machine learning application based on at least one annotated essay;
- and
- annotating said essay based on said probability,
- wherein said feature comprises a punctuation.
4 more elements
A computer readable medium on which is embedded computer software, said software comprising executable code for performing a method comprising:
- identifying a sentence of an essay;
- determining a feature associated with said sentence;
- determining a probability of said sentence being a discourse element by mapping said feature to a model, said model having been generated by a machine learning application based on at least one annotated essay;
- and
- annotating said essay based on said probability,
- wherein said feature comprises a lexical feature.
3 more elements
A computer readable medium on which is embedded computer software, said software comprising executable code for performing a method comprising:
- identifying a sentence of an essay;
- determining a feature associated with said sentence;
- determining a probability of said sentence being a discourse element by mapping said feature to a model, said model having been generated by a machine learning application based on at least one annotated essay;
- and
- annotating said essay based on said probability,
- wherein said feature comprises a rhetorical feature.
3 more elements
A computer readable medium on which is embedded computer software, said software comprising executable code for performing a method comprising:
- identifying a sentence of an essay;
- determining a feature associated with said sentence;
- determining a probability of said sentence being a discourse element by mapping said feature to a model, said model having been generated by a machine learning application based on at least one annotated essay;
- and
- annotating said essay based on said probability,
- wherein said feature comprises a punctuation.
3 more elements
An automatic essay annotator comprising:
- means for identifying a sentence of an essay;
- means for determining a feature associated with said sentence;
- means for determining a probability of said sentence being a discourse element, said means for determining said probability being configured to map said feature to a model, said model having been generated by a machine learning application based an at least one annotated essay and said discourse element being at least one of:
- title;
- background;
- thesis statement;
- main points;
- support;
- and conclusion;
- and
- means for annotating said essay based on said probability,
- wherein said means for determining said feature further comprises means for determining a lexical feature.
9 more elements
An automatic essay annotator comprising:
- means for identifying a sentence of an essay;
- means for determining a feature associated with said sentence;
- means for determining a probability of said sentence being a discourse element, said means for determining said probability being configured to map said feature to a model, said model having been generated by a machine learning application based on at least one annotated essay and said discourse element being at least one of:
- title;
- background;
- thesis statement;
- main points;
- support;
- and conclusion;
- and
- means for annotating said essay based on said probability,
- wherein said means for determining said feature further comprises means for determining a rhetorical feature.
9 more elements
An automatic essay annotator comprising:
- means for identifying a sentence of an essay;
- means for determining a feature associated with said sentence;
- means for determining a probability of said sentence being a discourse element, said means for determining said probability being configured to map said feature to a model, said model having been generated by a machine learning application based on at least one annotated essay and said discourse element being at least one of:
- title;
- background;
- thesis statement;
- main points;
- support;
- and conclusion;
- and
- means for annotating said essay based on said probability,
- wherein said means for determining said feature further comprises means for determining a punctuation.
9 more elements
Dependencies are inferred from the claim text.