Information processing apparatus, method and non-transitory computer-readable storage medium

A method executed by a computer, the computer comprising a display device, a line-of-sight (LOS) detection device mounted close to a periphery of the display device, the LOS detection device being configured to include an infrared light emitting diode (IR-LED) device and an infrared (IR) image sensor device to generate an image to be used for detecting a LOS of a user, and a memory configured to store plural pieces of question data, each of the plural pieces of question data including query data indicating a query sentence and answer data indicating an answer relating to the query sentence, the method comprising:

• obtaining first query data from any one of the plural pieces of question data to display the query sentence indicated by the obtained a first query data on the display device in a first display form, the query sentence indicated by the first query data including a question and a plurality of choices, each of the plurality of choices being a selection item to be used for answering to the question;
• during the displaying of the query sentence indicated by the first query data, obtaining a measurement result indicating the LOS of the user by measuring a relative position of a first gravity point to a second gravity point, the first gravity point being a gravity point of a pupil detected from the image generated by the IR image sensor device of the LOS detection device, the second gravity point being a gravity point of a Purkinje image detected from the image generated by the IR image sensor device of the LOS detection device;
• converting the obtained measurement result to behavior information indicating a change in an eye-gaze position on a screen of the display device by using a reference information indicating a plurality of data pairs each including a respective direction of the LOS and a corresponding eye-gaze position;
• in response to receiving an input of a first answer indicating a selection of a first choice being any one of the plurality of choices of the first query data displayed by the first display form, obtain a first confidence level for the received input by inputting a display area of the query sentence indicated by the first query data and the change in the eye-gaze position indicated by the behavior information into a pretrained model, the pretrained model being a machine learning model having been trained by using the display area of the query sentence and the eye-gaze position obtained in advance to output a higher score as a confidence level for an input of an answer is higher;
• in response to the first confidence level being less than a threshold value, converting the query sentence indicated by the first query data to a second display form different from the first display form to display the converted query sentence on the display device, the second display form being configured to replace a display position of the first choice indicated by the first answer with a display position of any one of the plurality choices other than the first choice;
• and
• in response to receiving an input of a second answer indicating a selection of any one of the plurality of choices included in the converted query sentence, obtaining a second confidence level by adjusting the first confidence level in accordance with whether the second answer for the first query data is identical to the answer indicated by the answer data corresponding to the first query data,
• wherein the adjusting of the first confidence level is configured to
• increase the first confidence level in response to the second answer being identical to the answer indicated by the answer data corresponding to the first query data to output the adjusted first confidence level as the second confidence level, and
• reduce the first confidence level in response to the second answer being not identical to the answer indicated by the answer data corresponding to the first query data to output the adjusted first confidence level as the second confidence level.

A non-transitory computer-readable storage medium storing a program that causes an information processing apparatus to execute a process, the information processing apparatus including a display device, a line-of-sight (LOS) detection device mounted close to a periphery of the display device, the LOS detection device being configured to include an infrared light emitting diode (IR-LED) device and an infrared (IR) image sensor device to generate an image to be used for detecting a LOS of a user, and a memory configured to store plural pieces of question data, each of the plural pieces of question data including query data indicating a query sentence and answer data indicating an answer relating to the query sentence, the process comprising:

• obtaining first query data from any one of the plural pieces of question data to display the query sentence indicated by the obtained first query data on the display device in a first display form, the query sentence indicated by the first query data including a question and a plurality of choices, each of the plurality of choices being a selection item to be used for answering to the question;
• during the displaying of the query sentence indicated by the first query data, obtaining a measurement result indicating the LOS of the user by measuring a relative position of a first gravity point to a second gravity point, the first gravity point being a gravity point of a pupil detected from the image generated by the IR image sensor device of the LOS detection device, the second gravity point being a gravity point of a Purkinje image detected from the image generated by the IR image sensor device of the LOS detection device;
• converting the obtained measurement result to behavior information indicating a change in an eye-gaze position on a screen of the display device by using a reference information indicating a plurality of data pairs each including a respective direction of the LOS and a corresponding eye-gaze position;
• in response to receiving an input of a first answer indicating a selection of a first choice being any one of the plurality of choices of the first query data displayed by the first display form, obtain a first confidence level for the received input by inputting a display area of the query sentence indicated by the first query data and the change in the eye-gaze position indicated by the behavior information into a pretrained model, the pretrained model being a machine learning model having been trained by using the display area of the query sentence and the eye-gaze position obtained in advance to output a higher score as a confidence level for an input of an answer is higher;
• in response to the first confidence level being less than a threshold value, converting the query sentence indicated by the first query data to a second display form different from the first display form to display the converted query sentence on the display device, the second display form being configured to replace a display position of the first choice indicated by the first answer with a display position of any one of the plurality choices other than the first choice;
• and
• in response to receiving an input of a second answer indicating a selection of any one of the plurality of choices included in the converted query sentence, obtaining a second confidence level by adjusting the first confidence level in accordance with whether the second answer for the first query data is identical to the answer indicated by the answer data corresponding to the first query data,
• wherein the adjusting of the first confidence level is configured to
• increase the first confidence level in response to the second answer being identical to the answer indicated by the answer data corresponding to the first query data to output the adjusted first confidence level as the second confidence level, and
• reduce the first confidence level in response to the second answer being not identical to the answer indicated by the answer data corresponding to the first query data to output the adjusted first confidence level as the second confidence level.