Problem collection/evaluation method, proposed solution collection/evaluation method, server for problem collection/evaluation, server for proposed solution collection/evaluation, and server for collection/evaluation of problem and proposed solution thereto

A method for collecting and evaluating proposed solutions, comprising:

• receiving, by a server, at least one problem to be solved selected from a plurality of problems in a problem data stored in a problem data storage part from a test administrator terminal via a network;
• receiving, by the server, an instruction to start a second test from the test administrator terminal via the network;
• extracting, by the server, the problem data including the at least one problem to be solved from the problem data storage part and extracting, by the server, a second format data for inputting a proposed solution to the at least one problem to be solved including at least one descriptive proposed solution input section from a second format data storage part in response to the instruction to start the second test, and transmitting, by the server, the problem data and the second format data to each of a plurality of examinee terminals in the second test via the network;
• receiving, by the server, a proposed solution data including a proposed solution to the at least one problem to be solved by the examinee in the second test from each of the examinee terminals in the second test;
• assigning, by the server, an identifier to each of the received proposed solution data including the proposed solution, and storing, by the server, the proposed solution data in a proposed solution data storage part in association with an identifier of the examinee in the second test who has transmitted the proposed solution data;
• acquiring, by the server, random numbers generated by a random number generator, and using, by the server, the random numbers to determine each examinee who should evaluate the proposed solution in each proposed solution data stored in the proposed solution data storage part from among the examinees in the second test;
• extracting, by the server, the proposed solution data including the proposed solution to be evaluated by each examinee from the proposed solution data storage part according to a result of the determining the examinees to evaluate the proposed solution, and extracting, by the server, a fourth format data for proposed solution evaluation input including a selective proposed solution evaluation input section based on at least one evaluation axis from a fourth format data storage part, and transmitting, by the server, the proposed solution data and the fourth format data to a corresponding examinee terminal in the second test via the network;
• receiving, by the server, a proposed solution evaluation data including an evaluation by the examinee in the second test of the proposed solution in the proposed solution data from each of the examinee terminals in the second test;
• assigning, by the server, an identifier to each of the received proposed solution evaluation data, and storing, by the server, the proposed solution evaluation data in a proposed solution evaluation data storage part in association with the identifier of the examinee in the second test as an evaluator who has transmitted the proposed solution evaluation data and the identifier of the evaluated proposed solution data;
• calculating, by the server, a score for each proposed solution for each evaluation axis by aggregating the evaluation of the proposed solution in each proposed solution data based on each proposed solution evaluation data and the identifier of the proposed solution data stored in the proposed solution evaluation data storage part, and storing, by the server, the score in the proposed solution data storage part in association with the identifier of the examinee in the second test who has transmitted the proposed solution data including the proposed solution;
• and
• extracting, by the server, a proposed solution evaluation aggregated data including each proposed solution and the score for each proposed solution for each evaluation axis stored in the proposed solution data storage part, and transmitting, by the server, the proposed solution evaluation aggregated data to the test administrator terminal via the network;
• calculating, by the server, an acquired score for each evaluation axis for each identifier of the examinee in the second test who has transmitted the proposed solution data at least partially based on the score for the proposed solution for each evaluation axis stored in the proposed solution data storage part in association with the identifier of the proposed solution data and the identifier of the examinee in the second test who has transmitted the proposed solution data, and storing, by the server, the acquired score in a second test examinee evaluation data storage part in association with the identifier of the examinee in the second test who has transmitted the proposed solution data;
• ranking, by the server, each examinee in the second test based on the identifier of the examinee in the second test and the acquired score associated with the identifier of the examinee, and re-aggregating and scoring, by the server, the evaluation of the proposed solution for each evaluation axis for each identifier of the proposed solution data to obtain a corrected score on condition that a higher weight is given to the evaluation as the evaluation is associated with the identifier of a higher-ranked examinee, and storing, by the server, the obtained corrected score for each evaluation axis in the proposed solution data storage part in association with the identifier of the proposed solution data;
• repeating at least once:
• calculating by the server, a corrected acquired score for each examinee for each evaluation axis in the second test based on the corrected score given to each proposed solution for each evaluation axis and the identifier of the examinee in the second test stored in the proposed solution data storage part and storing, by the server, the corrected acquired score in the second test examinee evaluation data storage part in association with the identifier of the examinee in the second test;
• and
• ranking, by the server, each examinee in the second test based on the identifier of the examinee in the second test and the corrected acquired score associated with the identifier of the examinee, and re-aggregating and scoring, by the server, the evaluation of the proposed solution for each evaluation axis for each identifier of the proposed solution data to obtain a corrected score on condition that a higher weight is given to the evaluation as the evaluation is associated with the identifier of a higher-ranked examinee, and storing, by the server, the obtained corrected score for each evaluation axis in the proposed solution data storage part in association with the identifier of the proposed solution data.

A server for collecting and evaluating problems, the server for collecting and evaluating problems comprising:

• a central processing unit (CPU), a transceiver, a control unit, a storage unit, and a random number generator,
• the storage unit comprising:
• a first test examinee account data storage part for storing an account information of each examinee in a first test in association with an identifier of the examinee;
• a first format data storage part for storing a first format data for problem input including at least one descriptive problem input section;
• a third format data storage part for storing a third format data for problem evaluation input including a selective problem evaluation input section based on at least one evaluation axis;
• a problem data storage part for storing a problem data including a problem in the descriptive problem input section input by each examinee in the first test received by the transceiver, and an identifier of the problem data, in association with the identifier of the examinee in the first test who has transmitted the problem data;
• and
• a problem evaluation data storage part for storing a problem evaluation data including an evaluation by the examinee of each problem received by the transceiver, and an identifier of the problem evaluation data, in association with the identifier of the examinee in the first test as an evaluator who has transmitted the problem evaluation data and the identifier of the evaluated problem data;
• the control unit comprising a data registration part, a problem input format extraction part, an evaluator determination part, a problem extraction part, a problem evaluation part, and a problem evaluation extraction part, wherein:
• the data registration part assigns an identifier to each of the problem data received at the transceiver and store the problem data in the problem data storage part in association with the identifier of the examinee in the first test who has transmitted the problem data, and assigns an identifier to each of the problem evaluation data received at the transceiver and store the problem evaluation data in the problem evaluation data storage part in association with the identifier of the examinee in the first test as an evaluator who has transmitted the problem evaluation data and the identifier of the evaluated problem data;
• the problem input format extraction part extracts the first format data for problem input from the first format data storage part and transmits the first format data to each of a plurality of examinee terminals in the first test from the transceiver via a network when the transceiver receives an instruction from a test administrator terminal;
• the evaluator determination part acquires random numbers generated by the random number generator, and uses the random numbers to determine each examinee who should evaluate the problem in each problem data stored in the problem data storage part by associating the identifier of the examinee in the first test as an evaluator with each identifier of the problem data when the transceiver receives an instruction from the test administrator terminal;
• the problem extraction part extracts the problem data including the problem to be evaluated by each examinee based on the identifier of the problem data and the identifier of the examinee in the first test as an evaluator associated with the identifier of the problem data according to the determination of the examinee for evaluation by the evaluator determination part, and extracts the third format data for problem evaluation input from the third format data storage part, and transmits the problem data and the third format data to a corresponding examinee terminal in the first test from the transceiver via the network;
• the problem evaluation part calculates a score for each problem for each evaluation axis by aggregating the evaluation of the problem in each problem data based on each problem evaluation data stored in the problem evaluation data storage part and the identifier of the problem data, and stores the obtained score in the problem data storage part in association with the identifier of the problem data;
• the problem evaluation extraction part extracts a problem evaluation aggregated data including each problem and the score for each problem for each evaluation axis stored in the problem data storage part, and transmits the problem evaluation aggregated data to the test administrator terminal via the network;
• the storage unit further comprises a first test examinee evaluation data storage part for storing an acquired score for the examinee in the first test in association with the identifier of the examinee;
• the control unit further comprises a first test examinee evaluation part which calculates the acquired score for each evaluation axis for each identifier of the examinee in the first test who has transmitted the problem data at least partially based on the score for the problem for each evaluation axis stored in the problem data storage part in association with the identifier of the problem data and the identifier of the examinee in the first test who has transmitted the problem data, and stores the acquired score in the first test examinee evaluation data storage part in association with the identifier of the examinee in the first test who has transmitted the problem data;
• and
• the problem evaluation part further ranks each examinee in the first test based on the identifier of the examinee in the first test and the acquired score associated with the identifier of the examinee, and re-aggregates and scores the evaluation of the problem for each evaluation axis for each identifier of the problem data to obtain a corrected score on condition that a higher weight is given to the evaluation as the evaluation is associated with the identifier of a higher-ranked examinee, and stores the obtained corrected score for each evaluation axis in the problem data storage part in association with the identifier of the problem data
• repeating at least once:
• the first test examinee evaluation part calculates a corrected acquired score for each examinee for each evaluation axis in the first test based on the corrected score given to each problem for each evaluation axis and the identifier of the examinee in the first test stored in the problem data storage part, and stores the corrected acquired score in the first test examinee evaluation data storage part in association with the identifier of the examinee in the first test;
• and
• the problem evaluation part ranks, each examinee in the first test based on the identifier of the examinee in the first test and the corrected acquired score associated with the identifier of the examinee, and re-aggregates and scores the evaluation of the problem for each evaluation axis for each identifier of the problem data to obtain a corrected score on condition that a higher weight is given to the evaluation as the evaluation is associated with the identifier of a higher-ranked examinee, and stores, the obtained corrected score for each evaluation axis in the problem data storage part in association with the identifier of the problem data.

A server for collecting and evaluating proposed solutions, the server comprising:

• a central processing unit (CPU), a transceiver, a control unit, a storage unit, and a random number generator,
• the storage unit comprising:
• a second examinee account data storage part for storing an account information of each examinee in a second test in association with an identifier of the examinee;
• a second format data storage part for storing a second format data for proposed solution input including at least one descriptive proposed solution input section;
• a fourth format data storage part for storing a fourth format data for proposed solution evaluation input including a selective proposed solution evaluation input section based on at least one evaluation axis;
• a problem data storage part for storing a problem data including a plurality of problems;
• a proposed solution data storage part for storing a proposed solution data including a proposed solution in the descriptive proposed solution input section input by each examinee in the second test received by the transceiver, and an identifier of the proposed solution data, in association with the identifier of the examinee in the second test who has transmitted the proposed solution data;
• and
• a proposed solution evaluation data storage part for storing a proposed solution evaluation data including an evaluation by the examinee of each proposed solution received by the transceiver, and an identifier of the proposed solution evaluation data, in association with the identifier of the examinee in the second test as an evaluator who has transmitted the proposed solution evaluation data and the identifier of the evaluated proposed solution data;
• the control unit comprising a data registration part, a proposed solution input format extraction part, an evaluator determination part, a proposed solution extraction part, a proposed solution evaluation part, and a proposed solution evaluation extraction part, wherein:
• the data registration part assigns an identifier to each of the proposed solution data received at the transceiver and store the proposed solution data in the proposed solution data storage part in association with the identifier of the examinee in the second test who has transmitted the proposed solution data, and assigns an identifier to each of the proposed solution evaluation data received at the transceiver and stores the proposed solution evaluation data in the proposed solution evaluation data storage part in association with the identifier of the examinee in the second test as an evaluator who has transmitted the proposed solution evaluation data and the identifier of the evaluated proposed solution data;
• the proposed solution input format extraction part extracts the problem data including at least one problem to be solved selected from a plurality of problems in the problem data from the problem data storage part, and extracts the second format data for proposed solution input from the second format data storage part, and transmits them to each of a plurality of examinee terminals in the second test from the transceiver via the network when the transceiver receives an instruction from a test administrator terminal,
• the evaluator determination part acquires random numbers generated by the random number generator, and uses the random numbers to determine each examinee who should evaluate the proposed solution in each proposed solution data stored in the proposed solution data storage part by associating the identifier of the examinee in the second test as an evaluator with each identifier of the proposed solution data when the transceiver receives an instruction from the test administrator terminal;
• the proposed solution extraction part extracts the proposed solution data including the proposed solution to be evaluated by each examinee based on the identifier of the proposed solution data and the identifier of the examinee in the second test as an evaluator associated with the identifier of the proposed solution data according to the determination of the examinee for evaluation by the evaluator determination part, and extracts the fourth format data for proposed solution evaluation input from the fourth format data storage part, and transmits the proposed solution data and the fourth format data to a corresponding examinee terminal in the second test from the transceiver via the network;
• the proposed solution evaluation part calculates a score for each evaluation axis for each proposed solution by aggregating the evaluation of the proposed solution in each proposed solution data based on each proposed solution evaluation data stored in the proposed solution evaluation data storage part and the identifier of the proposed solution data, and stores the obtained score in the proposed solution data storage part in association with the identifier of the proposed solution data;
• the proposed solution evaluation extraction part extracts a proposed solution evaluation aggregated data including each proposed solution and the score for each proposed solution for each evaluation axis stored in the proposed solution data storage part, and transmits the proposed solution evaluation aggregated data to the test administrator terminal via the network;
• the storage unit further comprises a second test examinee evaluation data storage part for storing an acquired score for each examinee in the second test in association with the identifier of the examinee;
• the control unit further comprises a second test examinee evaluation part which calculates the acquired score for each evaluation axis for each identifier of the examinee in the second test who has transmitted the proposed solution data at least partially based on the score for the proposed solution for each evaluation axis stored in the proposed solution data storage part in association with the identifier of the proposed solution data and the identifier of the examinee in the second test who has transmitted the proposed solution data, and stores the acquired score in the second test examinee evaluation data storage part in association with the identifier of the examinee in the second test who has transmitted the proposed solution data;
• the proposed solution evaluation part further ranks each examinee in the second test based on the identifier of the examinee in the second test and the acquired score associated with the identifier of the examinee, and re-aggregates and scores the evaluation of the proposed solution for each evaluation axis for each identifier of the proposed solution data to obtain a corrected score on condition that a higher weight is given to the evaluation as the evaluation is associated with the identifier of a higher-ranked examinee, and stores the obtained corrected score in the proposed solution data storage part in association with the identifier of the proposed solution data;
• repeating at least once:
• the second test examinee evaluation part calculates a corrected acquired score for each examinee for each evaluation axis in the second test based on the corrected score given to each proposed solution for each evaluation axis and the identifier of the examinee in the second test stored in the proposed solution data storage part, and stores the corrected acquired score in the second test examinee evaluation data storage part in association with the identifier of the examinee in the second test;
• and
• the proposed solution evaluation part ranks, each examinee in the second test based on the identifier of the examinee in the second test and the corrected acquired score associated with the identifier of the examinee, and re-aggregates and scores the evaluation of the proposed solution for each evaluation axis for each identifier of the proposed solution data to obtain a corrected score on condition that a higher weight is given to the evaluation as the evaluation is associated with the identifier of a higher-ranked examinee, and stores, the obtained corrected score for each evaluation axis in the proposed solution data storage part in association with the identifier of the proposed solution data.

A server for collecting and evaluating problems and proposed solutions thereto, the server comprising:

• a central processing unit (CPU), a transceiver, a control unit, a storage unit, and a random number generator,
• the storage unit comprising:
• a first test examinee account data storage part for storing an account information of each examinee in a first test in association with an identifier of the examinee;
• a second examinee account data storage part for storing an account information of each examinee in a second test in association with an identifier of the examinee;
• a first format data storage part for storing a first format data for problem input including at least one descriptive problem input section;
• a second format data storage part for storing a second format data for proposed solution input including at least one descriptive proposed solution input section;
• a third format data storage part for storing a third format data for problem evaluation input including a selective problem evaluation input section based on at least one evaluation axis;
• a fourth format data storage part for storing a fourth format data for proposed solution evaluation input including a selective proposed solution evaluation input section based on at least one evaluation axis;
• a problem data storage part for storing a problem data including a problem in the descriptive problem input section input by each examinee in the first test received by the transceiver, and an identifier of the problem data, in association with the identifier of the examinee in the first test who has transmitted the problem data;
• a proposed solution data storage part for storing a proposed solution data including a proposed solution in the descriptive proposed solution input section input by each examinee in the second test received by the transceiver, and an identifier of the proposed solution data, in association with the identifier of the examinee in the second test who has transmitted the proposed solution data;
• a problem evaluation data storage part for storing a problem evaluation data including an evaluation by the examinee of each problem received by the transceiver, and an identifier of the problem evaluation data, in association with the identifier of the examinee in the first test as an evaluator who has transmitted the problem evaluation data and the identifier of the evaluated problem data;
• and
• a proposed solution evaluation data storage part for storing a proposed solution evaluation data including an evaluation by the examinee of each proposed solution received by the transceiver, and an identifier of the proposed solution evaluation data, in association with the identifier of the examinee in the second test as an evaluator who has transmitted the proposed solution evaluation data and the identifier of the evaluated proposed solution data;
• the control unit comprising a data registration part, a problem input format extraction part, a proposed solution input format extraction part, an evaluator determination part, a problem extraction part, a proposed solution extraction part, a problem evaluation part, a proposed solution evaluation part, a problem evaluation extraction part, and a proposed solution evaluation extraction part, wherein;
• the data registration part assigns an identifier to each of the problem data received at the transceiver and stores the problem data in the problem data storage part in association with the identifier of the examinee in the first test who has transmitted the problem data, and assigns an identifier to each of the problem evaluation data received at the transceiver and stores the problem evaluation data in the problem evaluation data storage part in association with the identifier of the examinee in the first test as an evaluator who has transmitted the problem evaluation data and the identifier of the evaluated problem data;
• the data registration part is further assigns an identifier to each of the proposed solution data received at the transceiver and stores the proposed solution data in the proposed solution data storage part in association with the identifier of the examinee in the second test who has transmitted the proposed solution data, assigns an identifier to each of the proposed solution evaluation data received at the transceiver and stores the proposed solution evaluation data in the proposed solution evaluation data storage part in association with the identifier of the examinee in the second test as an evaluator who has transmitted the proposed solution evaluation data and the identifier of the evaluated proposed solution data;
• the problem input format extraction part extracts the first format data for problem input from the first format data storage part and transmits the first format data to each of a plurality of examinee terminals in the first test from the transceiver via a network when the transceiver receives an instruction from a test administrator terminal;
• the proposed solution input format extraction part extracts the problem data including at least one problem to be solved from the problem data storage part, and extracts the second format data for proposed solution input from the second format data storage part, and transmits the problem data and the second format data to each of a plurality of examinee terminals in the second test from the transceiver via the network when the transceiver receives an instruction from a test administrator terminal;
• the evaluator determination part acquires random numbers generated by the random number generator, and uses the random numbers to determine each examinee who should evaluate the problem in each problem data stored in the problem data storage part by associating the identifier of the examinee in the first test as an evaluator with each identifier of the problem data when the transceiver receives an instruction from the test administrator terminal;
• further, the evaluator determination part acquires random numbers generated by the random number generator, and uses the random numbers to determine each examinee who should evaluate the proposed solution in each proposed solution data stored in the proposed solution data storage part by associating the identifier of the examinee in the second test as an evaluator with each identifier of the proposed solution data when the transceiver receives an instruction from the test administrator terminal;
• the problem extraction part extracts the problem data including the problem to be evaluated by each examinee based on the identifier of the problem data and the identifier of the examinee in the first test as an evaluator associated with the identifier of the problem data according to the determination of the examinees for evaluation by the evaluator determination part, and extracts the third format data for problem evaluation input from the third format data storage part, and transmits the problem data and the third format data to a corresponding examinee terminal in the first test from the transceiver via the network;
• the proposed solution extraction part extracts the proposed solution data including the proposed solution to be evaluated by each examinee based on the identifier of the proposed solution data and the identifier of the examinee in the second test as an evaluator associated with the identifier of the proposed solution data according to the determination of the examinees for evaluation by the evaluator determination part, and extracts the fourth format data for proposed solution evaluation input from the fourth format data storage part, and transmits the proposed solution data and the fourth format data to a corresponding examinee terminal in the second test from the transceiver via the network;
• the problem evaluation part calculates a score for each problem for each evaluation axis by aggregating the evaluation of the problem in each problem data based on each problem evaluation data stored in the problem evaluation data storage part and the identifier of the problem data, and stores the obtained score in the problem data storage part in association with the identifier of the problem data;
• the proposed solution evaluation part calculates a score for each evaluation axis for each proposed solution by aggregating the evaluation of the proposed solution in each proposed solution data based on each proposed solution evaluation data stored in the proposed solution evaluation data storage part and the identifier of the proposed solution data, and stores the obtained score in the proposed solution data storage part in association with the identifier of the proposed solution data;
• the problem evaluation extraction part extracts a problem evaluation aggregated data including each problem and the score for each problem for each evaluation axis stored in the problem data storage part, and transmits the problem evaluation aggregated data to the test administrator terminal via the network;
• the proposed solution evaluation extraction part extracts a proposed solution evaluation aggregated data including each proposed solution and the score for each evaluation axis for each proposed solution stored in the proposed solution data storage part, and transmits the proposed solution evaluation aggregated data to the test administrator terminal via the network;
• the storage unit further comprises a second test examinee evaluation data storage part for storing an acquired score for each examinee in the second test in association with the identifier of the examinee;
• the control unit further comprises a second test examinee evaluation part which calculates the acquired score for each evaluation axis for each identifier of the examinee in the second test who has transmitted the proposed solution data at least partially based on the score for the proposed solution for each evaluation axis stored in the proposed solution data storage part in association with the identifier of the proposed solution data and the identifier of the examinee in the second test who has transmitted the proposed solution data, and stores the acquired score in the second test examinee evaluation data storage part in association with the identifier of the examinee in the second test who has transmitted the proposed solution data;
• and
• the proposed solution evaluation part further ranks each examinee in the second test based on the identifier of the examinee in the second test and the acquired score associated with the identifier of the examinee, and re-aggregates and scores the evaluation of the proposed solution for each evaluation axis for each identifier of the proposed solution data to obtain a corrected score on condition that a higher weight is given to the evaluation as the evaluation is associated with the identifier of a higher-ranked examinee;
• and stores the obtained corrected score in the proposed solution data storage part in association with the identifier of the proposed solution data;
• repeating at least once:
• the second test examinee evaluation part calculates a corrected acquired score for each examinee for each evaluation axis in the second test based on the corrected score given to each proposed solution for each evaluation axis and the identifier of the examinee in the second test stored in the proposed solution data storage part, and stores the corrected acquired score in the second test examinee evaluation data storage part in association with the identifier of the examinee in the second test;
• and
• the proposed solution evaluation part ranks, each examinee in the second test based on the identifier of the examinee in the second test and the corrected acquired score associated with the identifier of the examinee, and re-aggregates and scores the evaluation of the proposed solution for each evaluation axis for each identifier of the proposed solution data to obtain a corrected score on condition that a higher weight is given to the evaluation as the evaluation is associated with the identifier of a higher-ranked examinee, and stores, the obtained corrected score for each evaluation axis in the proposed solution data storage part in association with the identifier of the proposed solution data.