System and method for enabling crowd-sourced examination marking

A computer network implemented system for performing crowd sourced examination marking, the system comprising:

• one or more servers linked to the Internet and to a database, the one or more servers enabling one or more computer-implemented utilities providing:
• a marker community manager that enables authorized users of the system to access one or more features or functions for building and managing a plurality of markers associated with the system, and a profile manager for managing a plurality of marker profiles for each of the plurality of markers, and storing the plurality of marker profiles into the database;
• an examination content distributor for making accessible examination content to the plurality of markers using one or more network-connected devices, the content distributor enabling importing of scanned versions of paper based examination papers into the database such that the imported scanned versions are associated with an examination writer, and wherein an identity of the examination writer is withheld from the plurality of markers;
• and
• a marking manager that is configured to enable the authorized users to (i) create one or more active crowd sourced marking projects, and (ii) to automate the management of one or more workflows related to completion of one or more aspects of active crowd sourced marking projects;
• wherein the authorized users generate one or more marking parameters associated with marking projects or associated marking activities, where the marking parameters relating to marking quality criteria, and the marking parameters are stored to the database as a marking profile;
• a matching component for matching marking projects or marking activities to the plurality of markers, using the plurality of marker profiles, wherein the matching component:
• recommends markers from the plurality of markers for one or more of the marking projects,
• recommends allocation of a first marking activity to a first set of selected markers from the plurality of markers, and
• delivers examination content for marking including scoring and/or providing feedback on examination content by the first set of selected markers;
• wherein the identity of the examination writer in the imported scanned versions is represented with an indicia, wherein the indicia is a QR code,
• wherein each marker profile includes a trust rating based on at least one of competence and/or experience of the marker, wherein the one or more active crowd sourced marking projects includes a training portion comprising receiving a set of scored answers and feedback for the scored answers from the first set of selected markers, wherein each marker in the first set of selected markers has a trust score above a threshold, wherein the one or more active crowd sourced marking projects includes a crowd marking portion comprising distributing the examination content for marking to a second set of selected markers, wherein at least one of the markers in the second set of selected markers has a trust score below the threshold, and receiving a set of scored answers and feedback for the scored answers from the second set of selected markers.