Methods and systems for presentation and evaluation of constructed responses assessed by human evaluators

A method of analyzing the difficulty of questions administered in connection with a test and the performance of human evaluators in assessing constructed responses provided in response to the questions so administered, wherein the constructed responses are categorized based on the type of question to which the constructed response is responsive, the method comprising the steps of:

• identifying a first assessment station associated with a first human evaluator;
• identifying a second assessment station associated with a second human evaluator;
• electronically transmitting constructed responses of at least two categories to the first and the second assessment stations such that at least one of the categories transmitted to each of the first and the second assessment stations is the same;
• electronically receiving assessments for the constructed responses provided by the first and the second human evaluators from the first and the second assessment station, respectively, said assessments comprising a score for each constructed response;
• deriving a mean score awarded by said first human evaluator, a mean score awarded by said second human evaluator, and a mean score awarded to said at least one of the categories transmitted to each of the first and the second assessment stations;
• and
• comparing said mean score awarded by said first human evaluator, said mean score awarded by said second human evaluator, and said mean score awarded to said at least one of the categories transmitted to each of the first and the second assessment stations, to analyze the difficult of questions and the performance of human evaluators.