Loading patent…
AssigneeEducational Testing Service
InventorsIsaac I. Bejar, Sean J. Whalen
Estimating the term…
The present invention provides systems and methods for controlled presentation of constructed responses through a variety of computer applications to human evaluators for assessment. The systems and methods further provide for controlled presentation of the constructed responses to minimize the influences of psychometric factors on the accuracy of the human evaluators' assessments and to maximize the ability to efficiently generate data for use in analyzing the accuracy of the human evaluators' assessments and the difficulty of the constructed response categories or questions. The systems and methods of the present invention utilize a storage means which relationally stores data regarding the human evaluators, the constructed responses, the scores awarded and the computer applications to utilize. The systems also comprise assessment stations and processing means for selectively transmitting constructed responses to the assessment stations and receiving scores awarded by the human evaluators assigned to the assessment stations.
An application independent assessment system having a number of assessment stations at which human evaluators are assigned to assess constructed responses, wherein each constructed response is presentable at an assessment station using at least one computer application for the presentation of the constructed response, comprising:
A method of analyzing the difficulty of questions administered in connection with a test and the performance of human evaluators in assessing constructed responses provided in response to the questions so administered, wherein the constructed responses are categorized based on the type of question to which the constructed response is responsive, the method comprising the steps of:
A method of controlling presentation of constructed responses to human evaluators during an assessment session, wherein each constructed response is assigned to at least one constructed response category and is assessed by at least two human evaluators, the method comprising the steps of:
Dependencies are inferred from the claim text.