# US 2018/0322801 A1 · Computationally derived assessment in childhood education systems > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2018/0322801 A1 - Document ID: US20180322801A1 - Status: Published application - Kind: A1 - Application number: 15/587,022 - Filed: 2017-05-04 - Published: 2018-11-08 - Earliest claimed: 2017-05-04 - Assignees: International Business Machines Corporation - Applicants: International Business Machines Corporation - Inventors: Prasenjit Dey; Vijay Ekambaram; Ravindranath Kokku; Nitendra Rajput; Ruhi Sharma - Web page: https://patentgrep.com/patent/US20180322801A1 ## CPC - G09B 7/02: of the type wherein the student is expected to construct an answer to the question which is presented or wherein the machine gives an answer to the question presented by a student - G06F 17/18: for evaluating statistical data {, e.g. average values, frequency distributions, probability functions, regression analysis (forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04)} - G09B 5/06: with both visual and audible presentation of the material to be studied - G09B 5/065: {Combinations of audio and video presentations, e.g. videotapes, videodiscs, television systems} - G09B 5/08: providing for individual presentation of information to a plurality of student stations - G09B: EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS (devices for psychotechnics or for testing reaction times A61B5/16; games, sports, amusements A63; projectors, projector screens G03B) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A system for performing implicit assessment of a pupil is provided. The system receives a learning content from a content provider, the learning content having checkpoints at different stages of the learning content. The system associates assessment instructions with each checkpoint of the learning content. The system renders the received learning content for presentation to a pupil and captures the behaviors of the pupil during the presentation of the learning content. The system performs an assessment of the pupil at each checkpoint encountered during the rendering of the learning presentation. The assessment at a checkpoint includes using the checkpoint's associated assessment instructions to assign a score based on the captured behaviors. The system then produces an overall evaluation based on the assessments performed at the encountered checkpoints. ## Claims ### Claim 1 A computer-implemented method comprising: receiving a learning content data having checkpoints embedded at different stages of the learning content data; associating assessment instructions with each of the embedded checkpoints of the learning content data; rendering, at a processor, the received learning content data for presentation to a pupil and capturing a set of behaviors of the pupil using one or more sensors during the presentation of the learning content data; performing an assessment of the pupil at each of the embedded checkpoints encountered during the rendering of the learning presentation, wherein the assessment at an embedded checkpoint comprises using the embedded checkpoint's associated assessment instructions to assign a score based on the set of behaviors captured by the one or more sensors; and producing an overall evaluation based on the assessments performed at the encountered embedded checkpoints. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, further comprising injecting the assessment instructions into the learning content data for execution by the processor during the rendering of the learning content data. ### Claim 3 (depends on 1) The computer-implemented method of claim 1, wherein: each embedded checkpoint is associated with a priority weight; and producing the overall evaluation comprises weighting the assigned score of each embedded checkpoint according to the embedded checkpoint's associated priority weight. ### Claim 4 (depends on 3) The computer-implemented method of claim 3, wherein: the learning content data embeds at least a first checkpoint and a second checkpoint; and a priority weight associated with the first checkpoint is greater than a priority weight associated with the second checkpoint. ### Claim 5 (depends on 1) The computer-implemented method of claim 1, further comprising characterizing a captured behavior and assigning a confidence weight to the characterized behavior, wherein producing the overall evaluation comprises weighting the assigned score of an embedded checkpoint is based on the confidence weight of the characterized behaviors. ### Claim 6 (depends on 1) The computer-implemented method of claim 1, wherein using the embedded checkpoint's associated assessment instructions to assign a score based on the captured set of behaviors comprises using a threshold value that is normalized with respect to one or more characteristics of the pupil. ### Claim 7 (depends on 1) The computer-implemented method of claim 1, further comprising deriving the assessment instructions from statements written in a declarative language. ### Claim 8 (depends on 1) The computer-implemented method of claim 1, wherein the captured set of behaviors of the pupil comprise facial expressions, voices, gestures, eye-gazes, touches, and interactions with the presentation of the learning content data. ### Claim 9 (depends on 1) The computer-implemented method of claim 1, wherein: the assessment instructions of an embedded checkpoint comprise rules for classifying the captured set of behaviors as numerical values; and producing the overall evaluation comprises aggregating the numerical values of different embedded checkpoints into an overall evaluation value. ### Claim 10 A computing device, comprising: a processing unit; a set of sensors; and a storage device storing a set of instructions, wherein an execution of the set of instructions by the set of processing units configures the computing device to perform acts, comprising: receiving a learning content data having checkpoints embedded at different stages of the learning content data; associating assessment instructions with each embedded checkpoint of the learning content data; rendering the received learning content for presentation to a pupil and capturing a set of behaviors of the pupil using the set of sensors during the presentation of the learning content data; performing an assessment of the pupil at each embedded checkpoint encountered during the rendering of the learning presentation, wherein the assessment at an embedded checkpoint comprises using the embedded checkpoint's associated instructions to assign a score based on the set of behaviors captured by the set of sensors; and producing an overall evaluation based on the assessments performed at the encountered embedded checkpoints. ### Claim 11 (depends on 10) The computing device of claim 10, wherein the set of instructions further configures the computing device to inject the assessment instructions into the learning content data for execution during the rendering of the learning content data. ### Claim 12 (depends on 10) The computing device of claim 10, wherein: each embedded checkpoint is associated with a priority weight; producing the overall evaluation comprises weighting the assigned score of each embedded checkpoint according to the embedded checkpoint's associated priority weight; the learning content data embeds at least a first checkpoint and a second checkpoint; and a priority weight associated with the first checkpoint is greater than a priority weight associated with the second checkpoint. ### Claim 13 (depends on 10) The computing device of claim 10, wherein the set of instructions further configures the computing device to characterize a captured behavior and to assign a confidence weight to the characterized behavior, wherein producing the overall evaluation comprises weighting the assigned score of an embedded checkpoint is based on the confidence weight of the recognized behaviors. ### Claim 14 (depends on 10) The computing device of claim 10, wherein using the embedded checkpoint's associated assessment instructions to assign a score based on the captured set of behaviors comprises using a threshold value that is normalized with respect to one or more characteristics of the pupil. ### Claim 15 (depends on 10) The computing device of claim 10, wherein the captured set of behaviors of the pupil comprise facial expressions, voices, gestures, eye-gazes, touches, and interactions with the presentation of the learning content data. ### Claim 16 (depends on 10) The comprising device of claim 10, wherein: the assessment instructions of an embedded checkpoint comprise rules for classifying captured set of behaviors as numerical values; and producing the overall evaluation comprises aggregating the numerical values of different embedded checkpoints into an overall value. ### Claim 17 A computer-implemented method comprising: receiving a learning content data; receiving a set of behaviors that are captured using a set of sensors during presentation of the learning content data to a set of pupils; receiving a set of teacher evaluations, each teacher evaluation linking a captured behavior of a pupil with an assessment score at a stage of the learning content; and using, at a processor, the captured pupil behaviors and the received teacher evaluations as training sets to identify a set of checkpoints and to derive a set of assessment rules, wherein each checkpoint is associated with a stage of the learning content data and each assessment rule defines a scoring of a behavior at a checkpoint of the learning content. ### Claim 18 (depends on 17) The computer-implemented method of claim 17, further comprising using the captured pupil behaviors and the received teacher evaluations as training sets to identify a weighting factor for each checkpoint. ### Claim 19 (depends on 17) The computer-implemented method of claim 17, wherein the set of captured behaviors comprises characterized behaviors recognized from a video recording and an audio recording of the pupil. ### Claim 20 (depends on 17) The computer-implemented method of claim 17, wherein the derived set of assessment rules link pupil behaviors with understanding levels defined by the received set of teacher evaluations.