Method for online evaluation and online server for evaluation

A method for online evaluation, comprising:

• a step 1A in which a server allocates evaluators who should evaluate a plurality of evaluation targets that are stored in an evaluation target data storage part and are assigned identifiers, from among a plurality of evaluators who are assigned identifiers as evaluators for a current evaluation session;
• a step 1B in which the server extracts data related to the plurality of evaluation targets from the evaluation target data storage part according to a result of the step 1A, extracts question data related to a predetermined theme from a question data storage part, and extracts first format data for evaluation input including a selective evaluation input section based on at least one evaluation axis from a first format data storage part, and transmits the data related to the plurality of evaluation targets, the question data, and the first format data to corresponding terminals of the plurality of evaluators via a network;
• a step 1C in which the server receives evaluation result data including evaluations of the evaluation targets input by each evaluator in the selective evaluation input section, from the terminal of each evaluator via the network;
• a step 1 D in which the server assigns an identifier to each of the evaluation result data that have been received, and stores the evaluation result data in an evaluation result data storage part in association with the identifier of each evaluator who has transmitted the evaluation result data and the identifier of each evaluation target;
• a step 1E in which the server analyzes a degree of strictness of the evaluation by each evaluator for each evaluation axis based on the evaluation input in the selective evaluation input section by each evaluator in the evaluation result data stored in the evaluation result data storage part, and calculates a corrected evaluation by correcting the evaluation such that the evaluation by the evaluator who gives a strict evaluation rises relatively and the evaluation by the evaluator who gives a lax evaluation decreases relatively, and stores the corrected evaluation in the evaluation result data storage part in association with the identifier of each evaluator and the identifier of each evaluation target;
• a step 1F in which, the server performs for each of the first to nth evaluator, assuming that a number of the evaluators is n (n is an integer of 2 or more), without considering the evaluation of the evaluation target by the kth evaluator (k is an integer from 1 to n), aggregating the evaluations of each evaluation target based on the corrected evaluation by the evaluators other than the kth evaluator and the identifier of the evaluation target stored in the evaluation result data storage part to calculate a provisional score of each evaluation target for each evaluation axis, and storing the provisional score in an evaluation target score data storage part in association with the identifier of the kth evaluator and the identifier of each evaluation target;
• a step 1G1 in which the server performs for each of the first to nth evaluator comparing for each evaluation axis the corrected evaluation of each evaluation target associated with the identifier of the kth (k is an integer from 1 to n) evaluator stored in the evaluation result data storage part with the provisional score of each evaluation target stored in the evaluation target score data storage part associated with the identifier of the kth evaluator and the identifier of each evaluation target, aggregating closeness between them for each evaluator to calculate a provisional evaluation ability score of the kth evaluator, and storing the provisional evaluation ability score in the evaluator score data storage part in association with the identifier of the kth evaluator;
• a step 1H1 in which the server performs for each of the first to nth evaluator, without considering the evaluation of the evaluation target by the kth evaluator (k is an integer from 1 to n), aggregating the evaluations of each evaluation target based on the corrected evaluation by the evaluators other than the kth evaluator, the identifier of the evaluators and the identifier of the evaluation target stored in the evaluation result data storage part, and the provisional evaluation ability score of the evaluators other than the kth evaluator stored in the evaluator score data storage part to calculate a corrected score of each evaluation target for each evaluation axis, on condition that a greater weighting is given to the evaluation by the evaluator with a higher provisional evaluation ability score, and storing the corrected score in the evaluation target score data storage part in association with the identifier of the kth evaluator and the identifier of each evaluation target;
• a step 1G2 in which the server performs for each of the first to nth evaluator comparing for each evaluation axis the corrected evaluation of each evaluation target associated with the identifier of the kth (k is an integer from 1 to n) evaluator stored in the evaluation result data storage part with the corrected score of each evaluation target stored in the evaluation target score data storage part associated with the identifier of the kth evaluator and the identifier of each evaluation target, aggregating closeness between them for each evaluator to calculate a final evaluation ability score of the kth evaluator, and storing the final evaluation ability score in the evaluator score data storage part in association with the identifier of the kth evaluator;
• a step 1 H2 in which the server aggregates the evaluations of each evaluation target based on the corrected evaluation, the identifier of the evaluators and the identifier of the evaluation target stored in the evaluation result data storage part, and the final evaluation ability score of each evaluator stored in the evaluator score data storage part, to calculate a final score of each evaluation target for each evaluation axis, on condition that a greater weighting is given to the evaluation by the evaluator with a higher final evaluation ability score, and the server stores the final score in the evaluation target score data storage part in association with the identifier of each evaluation target;
• and
• a step 1I in which the server extracts either or both of the following data (1) and (2) and transmits them to a terminal of an administrator via the network:
• (1) data related to the evaluation targets, including the final score itself of each evaluation target for each evaluation axis and/or a statistic calculated based on the final score, stored in the evaluation target score data storage part.
• (2) data related to the evaluators, including the final evaluation ability score itself of each evaluator and/or a statistic calculated based on the final evaluation ability score, stored in the evaluator score data storage part.

A server for online evaluation, comprising a transceiver, a control unit, and a storage unit, wherein

• the storage unit comprises:
• an evaluation target data storage part for storing data related to a plurality of evaluation targets,
• a first format data storage part for storing first format data for evaluation input including a selective evaluation input section based on at least one evaluation axis;
• an evaluation result data storage part for storing evaluation result data including an evaluation and a corrected evaluation of each evaluation target;
• an evaluation target score data storage part for storing a provisional score and a corrected score of each evaluation target for each evaluation axis;
• an evaluator score data storage part for storing an evaluation ability score of each evaluator;
• the control unit comprises an evaluator allocation part, an evaluation input data extraction part, a data registration part, an evaluation analysis part, and an evaluation analysis data extraction part, wherein
• the evaluator allocation part is capable of performing a step 1A comprising allocating evaluators who should evaluate the plurality of evaluation targets that are stored in the evaluation target data storage part and are assigned identifiers, from among a plurality of evaluators who are assigned identifiers as evaluators of a current evaluation session,
• the evaluation input data extraction part is capable of performing a step 1B comprising extracting the data related to the plurality of evaluation targets from the evaluation target data storage part according to a result of the step 1A, extracting question data related to a predetermined theme from a question data storage part, extracting the first format data from the first format data storage part, and transmitting the data related to the plurality of evaluation targets, the question data, and the first format data to corresponding terminals of the plurality of evaluators via a network;
• the transceiver is capable of performing a step 1C comprising receiving the evaluation result data including evaluations of the evaluation targets input by each evaluator in the selective evaluation input section, from the terminal of each evaluator via the network, wherein
• the data registration part is capable of performing a step 1D comprising assigning an identifier to each of the evaluation result data that have been received, and storing the evaluation result data in the evaluation result data storage part in association with the identifier of each evaluator who has transmitted the evaluation result data and the identifier of each evaluation target;
• the evaluation analysis part is:
• capable of performing a step 1 E comprising analyzing a degree of strictness of the evaluation of each evaluator for each evaluation axis based on the evaluation input in the selective evaluation input section by each evaluator in the evaluation result data stored in the evaluation result data storage part, and calculating a corrected evaluation by correcting the evaluation such that the evaluation by the evaluator who gives a strict evaluation rises relatively and the evaluation by the evaluator who gives a lax evaluation decreases relatively, and storing the corrected evaluation in the evaluation result data storage part in association with the identifier of each evaluator and the identifier of each evaluation target;
• capable of performing a step 1F comprising aggregating the evaluations of each evaluation target based on the corrected evaluation and the identifier of the evaluation target stored in the evaluation result data storage part to calculate the provisional score of each evaluation target for each evaluation axis, and storing the provisional score in the evaluation target score data storage part in association with the identifier of each evaluation target;
• capable of performing a step 1G comprising comparing for each evaluation axis the corrected evaluation of each evaluation target associated with the identifier of the evaluator stored in the evaluation result data storage part with the provisional score of each evaluation target stored in the evaluation target score data storage part, aggregating closeness between them for each evaluator to calculate the evaluation ability score of each evaluator, and storing the evaluation ability score in the evaluator score data storage part in association with the identifier of each evaluator;
• capable of performing a step 1H comprising aggregating the evaluations for each evaluation target based on the corrected evaluation, the identifier of the evaluators and the identifier of the evaluation target stored in the evaluation result data storage part, and the evaluation ability score of each evaluator stored in the evaluator score data storage part, to calculate the corrected score of each evaluation target for each evaluation axis, on condition that a greater weighting is given to the evaluation by the evaluator with a higher evaluation ability score, and storing the corrected score in the evaluation target score data storage part in association with the identifier of each evaluation target;
• and
• the evaluation analysis data extraction part is capable of perform a step 1I comprising extracting either or both of the following data (1) and (2), and transmitting them from the transceiver to a terminal of an administrator via the network:
• (1) data related to the evaluation targets, including the corrected score itself of each evaluation target for each evaluation axis and/or a statistic calculated based on the corrected score, stored in the evaluation target score data storage part.
• (2) data related to the evaluators, including the evaluation ability score itself of each evaluator and/or a statistic calculated based on the evaluation ability score, stored in the evaluator score data storage part.

A server for online evaluation, comprising a transceiver, a control unit, and a storage unit, wherein

• the storage unit comprises:
• an evaluation target data storage part for storing data related to a plurality of evaluation targets,
• a first format data storage part for storing first format data for evaluation input including a selective evaluation input section based on at least one evaluation axis;
• an evaluation result data storage part for storing evaluation result data including an evaluation and a corrected evaluation of each evaluation target;
• an evaluation target score data storage part for storing a provisional score, a corrected score, and a final score of each evaluation target for each evaluation axis;
• an evaluator score data storage part for storing a provisional evaluation ability score and a final evaluation ability score of each evaluator;
• the control unit comprises an evaluator allocation part, an evaluation input data extraction part, a data registration part, an evaluation analysis part, and an evaluation analysis data extraction part, wherein
• the evaluator allocation part is capable of performing a step 1A comprising allocating evaluators who should evaluate the plurality of evaluation targets that are stored in the evaluation target data storage part and assigned identifiers, from among a plurality of evaluators who are assigned identifiers as evaluators of a current evaluation session,
• the evaluation input data extraction part is capable of performing a step 1B comprising extracting the data related to the plurality of evaluation targets from the evaluation target data storage part according to a result of the step 1A, extracting question data related to a predetermined theme from a question data storage part, extracting the first format data from the first format data storage part, and transmitting the data related to the plurality of evaluation targets, the question data, and the first format data to corresponding terminals of the plurality of evaluators via a network;
• the transceiver is capable of performing a step 1C comprising receiving the evaluation result data including the evaluations of the evaluation targets input by each evaluator in the selective evaluation input section, from the terminal of each evaluator via the network, wherein
• the data registration part is capable of performing a step 1D comprising assigning an identifier to each of the evaluation result data that have been received, and storing the evaluation result data in the evaluation result data storage part in association with the identifier of each evaluator who has transmitted the evaluation result data and the identifier of each evaluation target;
• the evaluation analysis part is:
• capable of performing a step 1 E comprising analyzing a degree of strictness of the evaluation of each evaluator for each evaluation axis based on the evaluation input in the selective evaluation input section by each evaluator in the evaluation result data stored in the evaluation result data storage part, and calculating a corrected evaluation by correcting the evaluation such that the evaluation by the evaluator who gives a strict evaluation rises relatively and the evaluation by the evaluator who gives a lax evaluation decreases relatively, and storing the corrected evaluation in the evaluation result data storage part in association with the identifier of each evaluator and the identifier of each evaluation target;
• capable of performing a step 1F comprising, for each of the first to nth evaluator, assuming that a number of the evaluators is n (n is an integer of 2 or more), without considering the evaluation of the evaluation target by the kth evaluator (k is an integer from 1 to n), aggregating the evaluations of each evaluation target based on the corrected evaluation by the evaluators other than the kth evaluator and the identifier of the evaluation target stored in the evaluation result data storage part to calculate a provisional score of each evaluation target for each evaluation axis, and storing the provisional score in the evaluation target score data storage part in association with the identifier of the kth evaluator and the identifier of each evaluation target;
• capable of performing a step 1G1 comprising, for each of the first to nth evaluator, comparing for each evaluation axis the corrected evaluation of each evaluation target associated with the identifier of the kth (k is an integer from 1 to n) evaluator stored in the evaluation result data storage part with the provisional score of each evaluation target stored in the evaluation target score data storage part associated with the identifier of the kth evaluator and the identifier of each evaluation target, aggregating closeness between them for each evaluator to calculate the provisional evaluation ability score of the kth evaluator, and storing the provisional evaluation ability score in the evaluator score data storage part in association with the identifier of the kth evaluator;
• capable of performing a step 1H1 comprising, for each of the first to nth evaluator, without considering the evaluation of the evaluation target by the kth evaluator (k is an integer from 1 to n), aggregating the evaluations of each evaluation target based on the corrected evaluation by the evaluators other than the kth evaluator, the identifier of the evaluators and the identifier of the evaluation target stored in the evaluation result data storage part, and the provisional evaluation ability score of the evaluators other than the kth evaluator stored in the evaluator score data storage part to calculate a corrected score of each evaluation target for each evaluation axis, on condition that a greater weighting is given to the evaluation by the evaluator with a higher provisional evaluation ability score, and storing the corrected score in the evaluation target score data storage part in association with the identifier of the kth evaluator and the identifier of each evaluation target;
• capable of performing a step 1G2 comprising, for each of the first to nth evaluator, comparing for each evaluation axis the corrected evaluation of each evaluation target associated with the identifier of the kth (k is an integer from 1 to n) evaluator stored in the evaluation result data storage part with the corrected score of each evaluation target stored in the evaluation target score data storage part associated with the identifier of the kth evaluator and the identifier of each evaluation target, aggregating closeness between them for each evaluator to calculate a final evaluation ability score of the kth evaluator, and storing the final evaluation ability score in the evaluator score data storage part in association with the identifier of the kth evaluator;
• capable of performing a step 1H2 comprising aggregating the evaluations of each evaluation target based on the corrected evaluation, the identifier of the evaluator and the identifier of the evaluation target stored in the evaluation result data storage part, and the final evaluation ability score of each evaluator stored in the evaluator score data storage part, to calculate the final score of each evaluation target for each evaluation axis, on condition that a greater weighting is given to the evaluation by the evaluator with a higher final evaluation ability score, and storing the final score in the evaluation target score data storage part in association with the identifier of each evaluation target;
• and
• the evaluation analysis data extraction part is capable of performing a step 1I comprising extracting either or both of the following data (1) and (2) and transmitting them from the transceiver to a terminal of an administrator via the network:
• (1) data related to the evaluation targets, including the final score itself of each evaluation target for each evaluation axis and/or a statistic calculated based on the final score, stored in the evaluation target score data storage part.
• (2) data related to the evaluators, including the final evaluation ability score itself of each evaluator and/or a statistic calculated based on the final evaluation ability score, stored in the evaluator score data storage part.