# US 2020/0401933 A1 · Closed loop biofeedback dynamic assessment

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0401933 A1
- Document ID: US20200401933A1
- Status: Published application
- Kind: A1
- Application number: 16/448,176
- Filed: 2019-06-21
- Published: 2020-12-24
- Earliest claimed: 2019-06-21
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: Jenna Reinen; Aldis Sipolins; Patrick Watson; Ravi Tejwani; Marco Cavallo; Hui Wu
- Web page: https://patentgrep.com/patent/US20200401933A1

## CPC

- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06N 20/00: Machine learning
- G06F 3/015: {Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection}
- G16C 60/00: Computational materials science, i.e. ICT specially adapted for investigating the physical or chemical properties of materials or phenomena associated with their design, synthesis, processing, characterisation or utilisation
- H04L 67/535: {Tracking the activity of the user (network monitoring arrangements H04L43/00; recording of computer activity G06F11/34)}
- G06F 2203/011: Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G16C: COMPUTATIONAL CHEMISTRY; CHEMOINFORMATICS; COMPUTATIONAL MATERIALS SCIENCE
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

A system includes a virtual reality system configured to enable a user to interact with a virtual environment, a plurality of biofeedback sensors configured to monitor a user, and a computer system including a virtual reality module configured to generate at least a view of the virtual environment, a biofeedback module configured to fuse output of the plurality of biofeedback sensors with a plurality of events within the virtual environment, a training module configured to generate a model of user behavior, wherein the training module executed by the computer system enables the computer system to make a prediction of a user response of the user based on a corpus of biofeedback data, and an alert module configured to generate at least one alert to the user via the virtual reality system based on the user response predicted by the computer system.

## Claims

### Claim 1

A system comprising:
a virtual reality system configured to enable a user to interact with a virtual environment;
a plurality of biofeedback sensors configured to monitor a user; and
a computer system comprising:
a virtual reality module configured to generate at least a view of the virtual environment;
a biofeedback module configured to fuse output of the plurality of biofeedback sensors with a plurality of events within the virtual environment;
a training module configured to generate a model of user behavior, wherein the training module executed by the computer system enables the computer system to predict a user response of the user based on a corpus of biofeedback data; and
an alert module configured to generate at least one alert to the user via the virtual reality system based on the user response predicted by the computer system.

### Claim 2 (depends on 1)

The system of claim 1, further comprising a means for connecting the computer system to a database of user data, wherein the training module is further configured to generate the model of user behavior using the database of user data.

### Claim 3 (depends on 1)

The system of claim 1, wherein the plurality of biofeedback sensors include at least one of a heart rate monitor, a electrocorticography sensor, a galvanic skin response sensor, and electromyogram sensor.

### Claim 4 (depends on 1)

The system of claim 1, further comprising a data logger storing the output of the plurality of biofeedback sensors.

### Claim 5 (depends on 1)

The system of claim 1, further comprising a data logger storing user activity within the virtual reality environment together with the output of the biofeedback sensors occurring a same time.

### Claim 6 (depends on 1)

The system of claim 1, wherein the alert is configured to require a response of the user.

### Claim 7 (depends on 6)

The system of claim 6, further comprising a data logger storing the response of the user together with the output of the biofeedback sensors prompting the alert.

### Claim 8 (depends on 1)

The system of claim 1, where the computer system generates a closed-feedback loop in which the training module and the biofeedback module are used to iteratively and adaptively update the model of user behavior.

### Claim 9

A method of constructing a predictive model for user behavior in a virtual environment comprising:
determining a baseline response for each of a plurality of sensors and a user;
building an individualized model comprising a plurality of priors generated from the baseline responses, the individualized model configured to predict a given response of the user based on a time-stamped stimulus presentation in the virtual environment;
sampling, iteratively, new responses of the user to respective events in the virtual environment during the time-stamped stimulus presentation;
linking each of the new responses and the respective events to an interval of time; and
updating at least one of the priors of the individualized model based on linked ones of the new responses and events.

### Claim 10 (depends on 9)

The method of claim 9, further comprising:
determining a threshold of predictability of the response of the user under a pre-determined context within the virtual environment;
receiving, during the sampling, output data from the sensors;
detecting in the output data that the threshold is satisfied; and
generating an alert to the virtual environment upon detecting that the threshold is satisfied.

### Claim 11 (depends on 10)

The method of claim 10, further comprising receiving a confirmation from the user based on the alert to the virtual environment.

### Claim 12 (depends on 10)

The method of claim 10, further comprising pausing the user's access to the virtual environment upon detecting that the threshold is satisfied.

### Claim 13 (depends on 10)

The method of claim 10, further comprising generating an alert to a remote device.

### Claim 14 (depends on 13)

The method of claim 13, further comprising:
pausing the user's access to the virtual environment upon detecting that the threshold is satisfied; and
resuming the user's access to the virtual environment upon receipt of a confirmation from the remote device.

### Claim 15

A non-transitory computer readable storage medium comprising computer executable instructions which when executed by a computer cause the computer to perform a method of constructing a predictive model for user behavior in a virtual environment, the method comprising:
determining a baseline response for each of a plurality of sensors and a user;
building an individualized model comprising a plurality of priors generated from the baseline responses, the individualized model configured to predict a given response of the user based on a time-stamped stimulus presentation in the virtual environment;
sampling, iteratively, new responses of the user to respective events in the virtual environment during the time-stamped stimulus presentation;
linking each of the new responses and the respective events to an interval of time; and
updating at least one of the priors of the individualized model based on linked ones of the new responses and events.

### Claim 16 (depends on 15)

The computer readable storage medium of claim 15, further comprising:
determining a threshold of predictability of the response of the user under a pre-determined context within the virtual environment;
receiving, during the sampling, output data from the sensors;
detecting in the output data that the threshold is satisfied; and
generating an alert to the virtual environment upon detecting that the threshold is satisfied.

### Claim 17 (depends on 16)

The computer readable storage medium of claim 16, further comprising receiving a confirmation from the user based on the alert to the virtual environment.

### Claim 18 (depends on 16)

The computer readable storage medium of claim 16, further comprising pausing the user's access to the virtual environment upon detecting that the threshold is satisfied.

### Claim 19 (depends on 16)

The computer readable storage medium of claim 16, further comprising generating an alert to a remote device.

### Claim 20 (depends on 19)

The computer readable storage medium of claim 19, further comprising:
pausing the user's access to the virtual environment upon detecting that the threshold is satisfied; and
resuming the user's access to the virtual environment upon receipt of a confirmation from the remote device.