Evaluating a worker in performing crowd sourced tasks and providing in-task training through programmatically generated test tasks

A method for testing and controlling work quality in crowdsourcing jobs, the method comprising:

• selecting a crowdsourcing job, distributed through a processor based job distribution platform, with a known correct result determined by agreement of different workers;
• generating a test crowdsourcing job by manipulating the known correct result into an incorrect result based on frequently received worker errors;
• presenting the test crowdsourcing job for evaluation to a first worker who is unaware of being evaluated;
• evaluating, by the processor based job distribution platform, work quality of the first worker by determining whether the first worker has identified the test crowdsourcing job as having the inaccurate result that was manipulated from the known correct result;
• controlling, by the processor based job distribution platform, work quality of the first worker, by minimizing repeat views of test crowdsourcing jobs, by the first worker, to a value equal to a ratio of maximum allowed job completeness for the first worker and test adjusted tasks.