Crowdsourcing system with community learning

A method comprising:

• receiving task data associated with a labeling task and candidate labels assigned to the labeling task;
• carrying out an inference using the received candidate labels and task data to jointly infer worker characteristics and characteristics of worker communities, by assuming relationships between two or more workers and by relating the characteristics of the worker communities to the worker characteristics through an addition of noise;
• and
• inferring an aggregated label for the task, the aggregated label taking into account the inferred worker characteristics and the inferred characteristics of the worker communities.

A system comprising:

• inference processing hardware;
• a processor arranged to:
• utilize the inference processing hardware to infer worker characteristics and characteristics of worker communities of which two or more workers are members;
• receive, via a communication interface, candidate labels and task data associated with a labeling task, and
• send inference results obtained from the inference processing hardware, the inference results comprising an aggregated label for the labeling task, the aggregated label taking into account the inferred worker characteristics and the inferred characteristics of the worker communities, wherein the inference processing hardware is configured to infer an updated probability distribution for latent variables representing at least one of the characteristics of the worker communities, the worker characteristics, or the aggregated label.