# US 10,122,775 B2 · Systems and methods for assessment and optimization of real-time collaborative intelligence systems

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,122,775 B2
- Document ID: US10122775B2
- Status: Granted
- Kind: B2
- Application number: 14/859,035
- Filed: 2015-09-18
- Published: 2016-02-25
- Granted: 2018-11-06
- Earliest claimed: 2014-03-26
- Assignees: Unanimous A.I., Inc.
- Applicants: Unanimous A.I., Inc.
- Inventors: Louis B. Rosenberg
- Web page: https://patentgrep.com/patent/US10122775B2

## CPC

- H04L 65/403: Arrangements for multi-party communication, e.g. for conferences (data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15)
- E21B 47/13: by electromagnetic energy, e.g. radio frequency
- E21B 47/18: through the well fluid {, e.g. mud pressure pulse telemetry}
- G06F 3/0482: Interaction with lists of selectable items, e.g. menus
- G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence)
- G06F 3/04847: Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
- G06F 16/955: using information identifiers, e.g. uniform resource locators [URL]
- H04L 67/10: in which an application is distributed across nodes in the network (software deployment G06F8/60; multiprogramming arrangements G06F9/46)
- H04L 67/12: specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
- H04L 67/125: involving control of end-device applications over a network
- H04L 67/14: Session management (for real-time applications in data packet communications networks H04L65/1066)
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- E21B: EARTH OR ROCK DRILLING; OBTAINING OIL, GAS, WATER, SOLUBLE OR MELTABLE MATERIALS OR A SLURRY OF MINERALS FROM WELLS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Systems and methods for real-time collaborative computing and collective intelligence are disclosed. A collaborative application runs on a collaborative server connected to a plurality of computing devices. Collaborative sessions are run wherein a group of independent users, networked over the internet, collaboratively answer questions in real-time, thereby harnessing their collective intelligence. A user performance value is determined for each of a plurality of independent users in the group based on each user's participation as compared to other users in the collaborative group. A group cohesiveness score is determined that quantifies the group's overall collaborative effectiveness. In some embodiments, the group cohesiveness score and user performance values are used to adjust weighting factors that affect the relative impact of each of the plurality of users in the collaborative group. In some embodiments, adjusting the weighting factors is performed with the objective of increasing the effectiveness of the resulting collective intelligence.

## Claims

### Claim 1

A method for determining a user performance value of each user of a group of users participating in a plurality of real-time collaborative sessions over a time period, wherein each user of the group is associated with a computing device in networked communication with a collaboration server, wherein during each collaborative session each computing device in real-time repeatedly determines a user intent from user input to the computing device, the user intent having an angular direction, and sends the user intent to the collaboration server, the collaboration server performing the steps of:
receiving, during a first collaborative session, in real-time from each computing device the corresponding user intent;
determining, during the first collaborative session, in real-time a group intent, having an angular direction, from the plurality of real-time user intents received from the plurality of computing devices;
determining, during the first collaborative session, a degree of alignment between the angular direction of each user intent and the angular direction of the group intent;
determining, during the first collaborative session, the user performance value for each user based at least in part upon the degree of alignment between the user intent associated with that user and the group intent, whereby a user performance value is determined for each user for each of the multiple time steps;
determining, during the first collaborative session, a weighting value for each of the plurality of users based at least in part on the user performance value associated with the user;
receiving, during a second collaborative session including display on a display of each computing device of a graphical object in a location relative to a set of targets, in real-time from each computing device the corresponding user intent;
applying, during the second collaborative session, the weighting value to the corresponding user intent;
determining, during the second collaborative session, the real-time group intent based on the weighted user intents;
determining, during the second collaborative sessions, an updated graphical object location based on the plurality of weighted user inputs; and
sending, during the additional collaborative session, the updated graphical object location to the plurality of computing devices, whereby the updated graphical object location is displayed on each of the plurality of computing devices.

### Claim 2 (depends on 1)

The method for determining the user performance value of claim 1, wherein the user performance value has an upper bound, wherein the user performance value at the upper bound indicates that the user intent was substantially aligned with the group intent.

### Claim 3 (depends on 1)

The method for determining the user performance value of claim 1, wherein the user weighting value is determined such that the user weighting value increases the contribution of the user when the user intents and the group intents used to determine the user performance value are more aligned and decreases the contribution of the user when the user intents and group intents are less aligned.

### Claim 4 (depends on 1)

The method for determining the user performance value of claim 1, wherein the user performance value has a lower bound, wherein the user performance value at the lower bound indicates that the user intent was substantially misaligned with the group intent.

### Claim 5 (depends on 4)

The method for determining the user performance value of claim 4, wherein the user intent is determined to be substantially misaligned with the group intent when the degree of alignment is such that the user intent and group intent are substantially opposite.

### Claim 6 (depends on 1)

The method for determining the user performance value of claim 1, wherein an average user performance value for one user over a second time period including a plurality of selected time steps is computed by finding the mean of the user performance values for the plurality of selected time steps.

### Claim 7 (depends on 6)

The method for determining the user performance value of claim 6, wherein the second time period substantially coincides with the collaborative session time period over which the group collaboratively moves at least one of the at least one graphical object to a select an answer from a plurality of displayed choices in response to a displayed question.

### Claim 8 (depends on 6)

The method for determining the user performance value of claim 6, wherein the average user performance value is time-weighted such that selected time steps near the end of the second time period are more heavily weighted than selected time steps near the start of the second time period.

### Claim 9 (depends on 1)

The method for determining the user performance value of claim 1, wherein a number of credits are awarded to at least one user based at least in part on the user performance value of that user.

### Claim 10 (depends on 9)

The method for determining the user performance value of claim 9, wherein the credits are redeemable by the user for the right to ask a question to the group, said question to be answered through an additional collaborative session.

### Claim 11 (depends on 1)

he method for determining the user performance value of claim 1, further comprising the step of determining by the collaboration server of an ordered ranking of the plurality of users, the ordered ranking based at least in part upon at least one user performance value associated with each of the plurality of users.

### Claim 12 (depends on 1)

The method for determining the user performance value of claim 1, wherein an assessment of the at least one user is determined based at least in part upon the user performance value of each user assessed.

### Claim 13 (depends on 12)

The method for determining the user performance value of claim 12, wherein the assessment indicates where each assessed user is one of at least “flexible” or “entrenched”.

### Claim 14 (depends on 12)

The method for determining the user performance value of claim 12, wherein the assessment indicates where each assessed user is one of at least “constructive” or “destructive”.

### Claim 15 (depends on 1)

The method for determining the user performance value of claim 1, comprising ejecting at least one user from the group based at least in part upon one user performance value associated with that user.

### Claim 16 (depends on 1)

The method for determining the user performance value of claim 1, further comprising selectively allowing a new user to join the group based at least in part upon a previously generated user performance value associated with the new user, said previously generated user performance value indicating a performance level of the new user in at least one prior collaborative session.

### Claim 17

A method for determining a group cohesiveness score of a group of individual users participating in a collaborative control session over a time period, wherein each user of the group is associated with an individual computing device in networked communication with a collaboration server, wherein during the collaborative session each computing device in real-time repeatedly determines a user intent from user input to the computing device, the user intent having an angular direction, and sends the user intent to the collaboration server, the collaboration server during each of multiple time steps of the collaborative session performing the steps of:
receiving in real-time from each of the computing device the corresponding user intent;
determining in real-time a group intent, having an angular direction, from the plurality of real-time user intents received from the plurality of computing devices;
determining in real-time a degree of alignment between the angular direction of each user intent and the group intent;
determining an average of the degree of alignment for the group;
determining a running average of the degree of alignment for the collaborative session over a selection of the multiple time steps;
determining in real-time a group cohesiveness score based at least in part on the running average of the degree of alignment; and
modulating the make-up of the group by at least one selected from the group of actions comprising ejecting at least one user from the group based at least in part upon a determination that the ejected user's real-time user intents were poorly aligned with the group intents over a period of time and selectively allowing a new user to join the group based at least in part upon a previously generated user performance value associated with the new user, said previously generated user performance value indicating a performance level of the new user in at least one prior collaborative session.

### Claim 18 (depends on 17)

The method for determining the group cohesiveness score of claim 17 wherein the collaborative session includes the group of individual users responding to a question by moving at least one graphical object to one of a plurality of input choices displayed on each computing device.

### Claim 19 (depends on 17)

The method for determining the group cohesiveness score of claim 17 wherein the group cohesiveness score has an upper bound wherein the upper bound indicates that the plurality of real-time user intents are substantially aligned with each other.

### Claim 20 (depends on 17)

The method for determining the group cohesiveness score of claim 17, wherein the group cohesiveness score has a lower bound wherein the lower bound that indicates that the plurality of real-time user intents are substantially misaligned with each other.

### Claim 21 (depends on 20)

The method for determining the group cohesiveness score of claim 20, wherein the plurality of real-time user intents are determined to be substantially misaligned when a summation of the user intent directions results in a substantially zero resultant.

### Claim 22 (depends on 17)

The method for determining the group cohesiveness score of claim 17, further comprising the step of determining an average group cohesiveness score for the group of users for a second time period including a plurality of selected time steps by finding the mean of the group cohesiveness scores during the plurality of selected time steps.

### Claim 23 (depends on 22)

The method for determining the group cohesiveness score of claim 22, wherein the mean is time-weighted such that selected time steps near the end of the second time period are more heavily weighted than selected time steps near the start of the second time period.

### Claim 24 (depends on 22)

The method for determining the group cohesiveness score of claim 22, wherein the mean group cohesiveness score is included in a social media message that also includes an associated answer collaboratively selected by the group.

### Claim 25 (depends on 17)

The method for determining the group cohesiveness score of claim 17, wherein a number of credits are awarded to each of the individual users based at least in part on the group cohesiveness score.

### Claim 26 (depends on 25)

The method for determining the group cohesiveness score of claim 25, wherein the credits are redeemable by each of the users for the right to ask a question to the group of users, said question to be answered through an additional collaborative session.

### Claim 27 (depends on 17)

The method for determining the group cohesiveness score of claim 17, further comprising the step of ranking of the group of users with respect to additional groups of users based at least in part upon at least one group cohesiveness score.

### Claim 28 (depends on 17)

The method for determining the group cohesiveness score of claim 17, further comprising the step of providing an assessment of the group based at least in part on at least one of the group cohesiveness scores.

### Claim 29 (depends on 28)

The method for determining the group cohesiveness score of claim 28, wherein the assessment indicates where the group is one of at least “flexible” or “entrenched”.

### Claim 30 (depends on 17)

The method for determining the group cohesiveness score of claim 17, wherein the period of time includes the collaborative session and at least one additional collaborative session.

### Claim 31 (depends on 17)

The method for determining the group cohesiveness score of claim 17, further comprising the step of repeatedly reporting and repeatedly displaying the group cohesiveness score to each of the plurality of computing devices.

### Claim 32 (depends on 17)

The method for determining the group cohesiveness score of claim 17, wherein the group cohesiveness score is expressed as a percentage from 0% to 100%.