# US 2012/0083675 A1 · Measuring affective data for web-enabled applications

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2012/0083675 A1
- Document ID: US20120083675A1
- Status: Published application
- Kind: A1
- Application number: 13/249,317
- Filed: 2011-09-30
- Published: 2012-04-05
- Earliest claimed: 2010-09-30
- Inventors: Rana el Kaliouby; Rosalind Wright Picard; Richard Scott Sadowsky
- Web page: https://patentgrep.com/patent/US20120083675A1

## CPC

- A61B 5/165: {Evaluating the state of mind, e.g. depression, anxiety}
- A61B 5/163: {by tracking eye movement, gaze, or pupil change}
- G06Q 30/0271: {Personalized advertisement}
- G16H 20/70: relating to mental therapies, e.g. psychological therapy or autogenous training
- G16H 40/67: for remote operation
- A61B 5/02055: {Simultaneously evaluating both cardiovascular condition and temperature}
- A61B 5/02405: {Determining heart rate variability}
- A61B 5/0533: Measuring galvanic skin response
- A61B 5/08: Measuring devices for evaluating the respiratory organs
- A61B 5/11: Measuring movement of the entire body or parts thereof, e.g. head or hand tremor or mobility of a limb {(A61B5/1038 takes precedence; motion detection to correct for motion artifacts in physiological signals A61B5/721)}
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA

## Abstract

Mental state information is collected as a person interacts with a rendering such as a website or video. The mental state information is collected through video capture or capture of sensor information. The sensor information can be of electrodermal activity, accelerometer readings, skin temperature, or other characteristics. The mental state information is uploaded to a server and aggregated with other people's information so that collective mental states are associated with the rendering. The aggregated mental state information is displayed through a visual representation such as an avatar.

## Claims

### Claim 1

A computer implemented method for analyzing web-enabled application traffic comprising:
collecting mental state data from a plurality of people as they interact with a rendering;
uploading information, to a server, based on the mental state data from the plurality of people who interact with the rendering;
receiving aggregated mental state information on the plurality of people who interact with the rendering; and
displaying the aggregated mental state information with the rendering.

### Claim 2 (depends on 1)

The method of claim 1 wherein the aggregated mental state information includes norms derived from the plurality of people.

### Claim 3 (depends on 2)

The method of claim 2 wherein the norms are based on contextual information.

### Claim 4 (depends on 1)

The method of claim 1 further comprising associating the aggregated mental state information with the rendering.

### Claim 5 (depends on 1)

The method according to claim 1 wherein the rendering is one of a group comprising a button, an advertisement, a banner ad, a drop down menu, and a data element on a web-enabled application.

### Claim 6 (depends on 1)

The method according to claim 1 wherein the rendering is one of a group comprising a landing page, a checkout page, a webpage, a website, a web-enabled application, a video on a web-enabled application, a game on a web-enabled application, and a virtual world.

### Claim 7 (depends on 1)

The method according to claim 1 wherein the collecting mental state data involves capturing of one of a group comprising physiological data and facial data.

### Claim 8 (depends on 7)

The method according to claim 7 wherein a webcam is used to capture one or more of the facial data and the physiological data.

### Claim 9 (depends on 8)

The method according to claim 8 wherein the physiological data is used to determine autonomic activity.

### Claim 10 (depends on 9)

The method according to claim 9 wherein the autonomic activity is one of a group comprising heart rate, respiration, and heart rate variability.

### Claim 11 (depends on 7)

The method according to claim 7 wherein the facial data includes information on one or more of a group comprising facial expressions, action units, head gestures, smiles, brow furrows, squints, lowered eyebrows, raised eyebrows, and attention.

### Claim 12 (depends on 1)

The method according to claim 1 further comprising tracking of eyes to identify the rendering with which interacting is accomplished.

### Claim 13 (depends on 12)

The method according to claim 12 wherein the tracking of the eyes identifies a portion of the rendering on which the eyes are focused.

### Claim 14

(canceled)

### Claim 15 (depends on 12)

The method according to claim 12 further comprising recording of eye dwell time on the rendering and associating information on the eye dwell time to the rendering and to the mental state data.

### Claim 16

(canceled)

### Claim 17 (depends on 1)

The method according to claim 1 further comprising opting in, by an individual from the plurality of people, to allowing facial information to be aggregated.

### Claim at position 18

18-19. (canceled)

### Claim 20 (depends on 1)

The method according to claim 1 wherein aggregation of the aggregated mental state information is performed on a demographic basis so that mental state information is grouped based on the demographic basis.

### Claim 21 (depends on 1)

The method according to claim 1 further comprising creating a visual representation of one or more of the aggregated mental state information and mental state information on an individual from the plurality of people.

### Claim 22 (depends on 21)

The method according to claim 21 wherein the visual representation displays the aggregated mental state information on a demographic basis.

### Claim 23

(canceled)

### Claim 24 (depends on 1)

The method according to claim 1 further comprising synchronizing the aggregated mental state information with the rendering.

### Claim 25 (depends on 1)

The method according to claim 1 further comprising capturing contextual information about the rendering.

### Claim 26 (depends on 25)

The method according to claim 25 wherein the contextual information includes one or more of a timeline, a progression of web pages, and an actigraph.

### Claim 27 (depends on 1)

The method of claim 1 further comprising inferring of mental states based on the mental state data collected from the plurality of people.

### Claim 28 (depends on 27)

The method according to claim 27 wherein the mental states include one of a group comprising frustration, confusion, disappointment, hesitation, cognitive overload, focusing, being engaged, attending, boredom, exploration, confidence, trust, delight, and satisfaction.

### Claim 29

A computer program product embodied in a non-transitory computer readable medium for analyzing web-enabled application traffic, the computer program product comprising:
code for collecting mental state data from a plurality of people as they interact with a rendering;
code for uploading information, to a server, based on the mental state data from the plurality of people who interact with the rendering;
code for receiving aggregated mental state information on the plurality of people who interact with the rendering; and
code for displaying the aggregated mental state information with the rendering.

### Claim 30

A system for analyzing web-enabled application traffic states comprising:
a memory which stores instructions;
one or more processors attached to the memory wherein the one or more processors, when executing the instructions which are stored, are configured to:
collect mental state data from a plurality of people as they interact with a rendering;
upload information, to a server, based on the mental state data from the plurality of people who interact with the rendering;
receive aggregated mental state information on the plurality of people who interact with the rendering; and
display the aggregated mental state information with the rendering.

### Claim at position 30

31-33. (canceled)
