A/b testing for massive open online courses

A computer program product comprising a nontransitive storage medium, the computer program product including code that, when executed by processing circuitry of a computer, causes the processing circuitry to perform a method, the method comprising:

• massive open online course (MOOC) data and data describing a population of students enrolled in the MOOC, the MOOC data including data describing a plurality of learning modules of the MOOC, each of the plurality of learning modules including respective first course content and second course content;
• for each of the plurality of learning modules:
• assigning a first portion of the population of students to experience that learning module with the respective first course content and a second portion of the population of students to experience that learning module with the respective second course content;
• and
• performing a short-timescale evaluation operation on that learning module based on a specified metric applied to the first portion of the population of students and the second portion of the population of students to produce evaluation results for that learning module.

An electronic apparatus, comprising:

• memory;
• and
• controlling circuitry coupled to the memory, the controlling circuitry being configured to:
• obtain massive open online course (MOOC) data and data describing a population of students enrolled in the MOOC, the MOOC data including data describing a plurality of learning modules of the MOOC, each of the plurality of learning modules including respective first course content and second course content;
• for each of the plurality of learning modules:
• assign a first portion of the population of students to experience that learning module with the respective first course content and a second portion of the population of students to experience that learning module with the respective second course content;
• and
• perform a short-timescale evaluation operation on that learning module based on a specified metric applied to the first portion of the population of students and the second portion of the population of students to produce evaluation results for that learning module.