Method and system for interactive computer science testing, anaylsis and feedback

A method of administering to a student a problem in computer science for analysis and feedback comprising the steps of:

• a) providing an item pool to the student, said item pool comprising a plurality of predetermined test problems in computer science to be selected by the student;
• b) recording the student's solution program to a particular test problem selected;
• c) lexing the solution program into a list of tokens, each token being representative of a part of the solution program and its location within the solution program;
• d) parsing the list of tokens into an abstract syntax tree representation;
• e) comparing a predetermined pattern against a first portion of the abstract syntax tree;
• f) repeating the comparison step with a plurality of portions of the abstract syntax tree until the predetermined pattern matches a portion of the abstract syntax tree;
• g) repeating steps (e) and (f) for a plurality of predetermined patterns;
• h) providing to the student text messages correlated to the predetermined patterns so matched;
• whereby the student is automatically provided with information embodied in the text messages which is indicative of the correctness or incorrectness of the solution text entered by the student in response to the test problem selected.

A method of administering an examination in computer science to a student comprising the steps of:

• a) providing to the student a predetermined test problem in computer science from an item pool comprising a plurality of different predetermined test problems;
• b) recording the student's solution program in computer science to the predetermined test problem so provided;
• c) lexing the solution program into a list of tokens, each token being representative of a part of the solution program and its location within the solution program;
• d) parsing the list of tokens into an abstract syntax tree representation;
• e) comparing a predetermined pattern against a first portion of the abstract syntax tree;
• f) repeating the comparison step with a plurality of portions of the abstract syntax tree until the predetermined pattern matches a portion of the abstract syntax tree;
• g) repeating steps (e) and (f) for a plurality of predetermined patterns;
• and
• h) storing a score correlated to the predetermined patterns so matched;
• whereby the student's solution program to the predetermined test problem is automatically evaluated and scored, the score being indicative of the correctness or incorrectness of the solution text entered by the student in response to the test problem provided.

A computer system for administering a problem for analysis and feedback in a computer science programming language to a student, said system comprising:

• a) a user interface comprising
• i) an item pool comprising a plurality of predetermined test problems in computer science to be selected by the student;
• ii) output means for providing to the student the particular test problem selected;
• and
• iii) input means for providing communications from the student to the system;
• iv) editor means for recording to student's solution program in computer science to the particular test problem selected;
• b) a knowledge base comprising a set of predetermined knowledge base patterns;
• c) an analytic component comprising
• i) means for converting the student's solution program into an abstract syntax tree;
• ii) means for comparing each of the predetermined knowledge base patterns against portions of the abstract syntax tree until matched successfully;
• and
• d) means for providing to the output means of the user interface a text message correlated to the knowledge base patterns so matched;
• whereby the student is automatically provided with information embodies in the text message which is indicative of the correctness of the solution text entered by the student in response to the test problem selected.

An expert system for evaluating a solution program into to a computer in response to a computer science problem comprising:

• a) interface means for providing communication to and from a user;
• b) lexing means for converting said solution program into a list of tokens, each of said tokens being representative of a part of the solution program and its location within the solution program;
• c) parsing means for converting the list of tokens produced by the lexing means into an abstract syntax tree representation of the solution program;
• d) a knowledge base comprising a plurality of predetermined knowledge base patterns wherein each of said knowledge base patterns represent a certain part of a model solution program;
• e) an inference engine comprising means for comparing each of said knowledge base patterns against portions of the abstract syntax tree until successfully matched;
• and
• f) means for providing to the interface output messages correlated to the knowledge base patterns successfully matched.

A method for interactively compiling knowledge base patterns of computer science programs for matching against portions of abstract syntax tree representations of model solution programs in response to the computer science problems, said method comprising the steps of:

• a) lexing a model program into a list of tokens, each of said tokens being representative of a part of the model program and its location within the model program;
• b) parsing the list of tokens into an abstract syntax tree representation of the model program;
• c) querying a user to select the part of the model program against which the knowledge base pattern will be used to compare;
• and
• d) generating a knowledge base pattern in accordance with the user's selection such that said knowledge base pattern will match against a portion of the abstract syntax tree correlated to the part of the model program selected by the user.

A method of administering to a student problems in computer science for analysis and feedback comprising the steps of:

• a) providing an item pool to the student, said item pool comprising a plurality of predetermined test problems in computer science to be selected by the student;
• b) recording the student's solution program to a particular test problem selected;
• c) lexing the solution program into a list of tokens, each token being representative of a part of the solution program and its location within the solution program;
• d) parsing the list of tokens into an abstract syntax tree representation;
• e) comparing a predetermined pattern against a first portion of the abstract syntax tree;
• f) repeating the comparison step with a plurality of portions of the abstract syntax tree until the predetermined pattern matches a portion of the abstract syntax tree;
• g) repeating steps (e) and (f) for a plurality of predetermined patterns;
• h) providing to the student text messages correlated to the predetermined patterns so matched;
• and
• i) storing a score correlated to the predetermined patterns so matched;
• whereby the student is automatically provided with a score and information embodied in the text messages which is indicative of the correctness or incorrectness of the solution text entered by the student in response to the test problem selected.