Amplifying group intelligence by adaptive population optimization
AssigneeUnanimous A. I., Inc.
InventorsLOUIS B. ROSENBERG, GREGG WILLCOX
Life of this patent
Estimating the term…
Abstract
System and method for amplifying the accuracy of forecasts generated by software systems that harness the collective intelligence of human populations by curating optimized sub-populations through an intelligent selection process. Participants predict event outcomes and/or provide evaluations of their confidence in their predictions. The system determines a score wherein the alignment score indicates how well that participant's prediction aligns with the predictions given by the baseline population. Participants can then be selected from the population based on the participant alignment scores.
Claims16
A system for curating an optimized population of human forecasting participants from a baseline population of human forecasting participants, the system comprising:
- an analysis processing device including a processor and configured for network communication;
- and
- a plurality of application instances, wherein each application instance is configured for interacting with one participant and is in network communication with the analysis processing device, wherein the system is configured to perform the steps of:
- present, to each baseline population participant via the application instance, a set of possible outcomes for each of a set of future events, wherein each set of possible outcomes for each future event has at least two possible outcomes;
- collect a set of predictions from each baseline population participant via the application instance, wherein each set of predictions includes, for each future event in the set of future events, one predicted outcome selected from the set of possible outcomes for that future event, and wherein an actual outcome of each of the future events is unknown at the time of collecting;
- store the set of predictions from each baseline population participant;
- for each future event in the set of future events, compute, using the stored sets of predictions, at least one outcome support value for each possible outcome of that future event, wherein each outcome support value is based at least in part on a percentage of baseline population participants that predicted that possible outcome for that future event;
- for each baseline population participant, compute a set of alignment scores comprising an alignment score for each future event, wherein the alignment score for the future event is computed by algorithmically comparing the participant's predicted outcome for that future event to the outcome support value for that possible outcome of that future event, whereby the alignment score indicates how well that participant's prediction aligned with the predictions collected from the baseline population for that future event;
- for each baseline population participant, determine an alignment index based on the set of alignment scores computed for that participant, wherein the alignment index indicates how well the set of that participant's predictions aligned with the predictions collected from the baseline population;
- and
- curate an updated population of participants, the curating including at least one selected from the group of (a) culling a plurality of participants from the baseline population in response to the alignment index of each culled participant indicating low alignment compared to the baseline population, and (b) applying a weighting value to each of a set of baseline population participants, wherein the applied weighting values are lower for participants with the alignment index indicating low alignment with the baseline population and the applied weighting values are higher for baseline population participants with the alignment index indicating high alignment with the baseline population.
A method for curating an optimized population of human forecasting participants from a baseline population of human forecasting participants, comprising the steps of:
- presenting, by an analysis processing device including a processor and configured for networked communication, to each baseline population participant via an application instance in network communication with the analysis processing device and configured to interact with that participant, a set of possible outcomes for each of a set of future events, wherein each set of possible outcomes for each future event has at least two possible outcomes;
- collecting a set of predictions from each baseline population participant via the application instance, wherein each set of predictions includes, for each future event in the set of future events, one predicted outcome selected from the set of possible outcomes for that future event, and wherein an actual outcome of each of the future events is unknown at the time of collecting;
- storing the set of predictions from each baseline population participant;
- computing, by the analysis processing device for each future event in the set of future events, at least one outcome support value for each possible outcome of that future event, wherein each outcome support value is based at least in part on a percentage of baseline population participants that predicted that possible outcome for that future event;
- computing, by the analysis processing device for each baseline population participant, a set of alignment scores comprising an alignment score for each future event, wherein the alignment score for the future event is computed by algorithmically comparing the participant's predicted outcome for that future event to the outcome support value for that possible outcome of that future event, whereby the alignment score indicates how well participant's prediction aligned with the predictions collected from the baseline population for that future event;
- determining, by the analysis processing device for each baseline population participant, an alignment index based on the set of alignment scores computed for that participant, wherein the alignment index indicates how well the set of that participant's predictions aligned with the predictions collected from the baseline population;
- and
- curating, by the analysis processing device, an updated population of participants, based at least in part upon a plurality of the alignment indexes, the curating including at least one selected from the group of (a) culling a plurality of participants from the baseline population in response to the alignment index of each culled participant indicating low alignment compared to the baseline population, and (b) applying a weighting value to each of a set of baseline population participants, wherein the applied weighting values are lower for participants with the alignment index indicating low alignment with the baseline population and the applied weighting values are higher for baseline population participants with the alignment index indicating high alignment with the baseline population.
Dependencies are inferred from the claim text.