# US 2017/0285732 A1 · Queasiness management for virtual reality systems

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2017/0285732 A1
- Document ID: US20170285732A1
- Status: Published application
- Kind: A1
- Application number: 15/431,540
- Filed: 2017-02-13
- Published: 2017-10-05
- Earliest claimed: 2016-03-29
- Assignees: Dolby Laboratories Licensing Corporation
- Applicants: Dolby Laboratories Licensing Corporation
- Inventors: Scott DALY; Grant Haverstock MULLIKEN
- Web page: https://patentgrep.com/patent/US20170285732A1

## CPC

- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- A61B 5/0205: Simultaneously evaluating both cardiovascular conditions and different types of body conditions, e.g. heart and respiratory condition
- A61B 5/4836: {Diagnosis combined with treatment in closed-loop systems or methods (A61B5/0036 takes precedence)}
- A61B 5/742: {using visual displays (displays for heart-related electrical signals, e.g. ECG, A61B5/339)}
- G06F 3/012: {Head tracking input arrangements}
- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06F 3/015: {Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection}
- G06T 13/40: of characters, e.g. humans, animals or virtual beings
- G06T 15/20: Perspective computation
- G06T 2215/16: Using real world measurements to influence rendering
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Input VR imagery is received. Global motions as represented in the input VR imagery relative to a viewer of a virtual reality (VR) application is extracted. A dampening factor is applied to the global motions to generate dampened global motions. VR imagery to be rendered to the viewer at a time point is generated based on the input VR imagery and the dampened global motions.

## Claims

### Claim 1

A method, comprising:
receiving input VR imagery;
extracting global motions as represented in the input VR imagery relative to a viewer of a virtual reality (VR) application;
applying a dampening factor to the global motions to generate dampened global motions;
generating, based on the input VR imagery and the dampened global motions, VR imagery to be rendered to the viewer at a time point;
wherein the method is performed by one or more computing devices.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
extracting, from the viewer's motion sensor data, the viewer's ego-motions while the viewer is viewing the VR imagery;
correlating the global motions extracted from the input VR imagery with the viewer's ego motions extracted from the viewer's motion sensor data to generate frame-dependent motions;
applying the dampening factor to the frame-dependent motions to generate dampened frame-dependent motions.

### Claim 3 (depends on 2)

The method of claim 2, wherein the VR imagery at the time point is generated based on the input VR imagery and the dampened frame-dependent motions.

### Claim 4 (depends on 2)

The method of claim 2, wherein the VR imagery to be rendered at the time point is acquired or constructed based on the viewer's dampened ego-motions generated based on the dampened frame-dependent motions.

### Claim 5 (depends on 4)

The method of claim 4, wherein the input VR imagery represents undampened VR imagery acquired with a first image acquisition device at the viewer's represented undampened location and viewpoint; and wherein the VR imagery to be rendered at the time point represents dampened VR imagery acquired with a second image acquisition device at the viewer's represented dampened location and viewpoint.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
receiving one or more physiological monitoring indicators from the viewer up to a first time point;
determining, based at least in part on the one or more physiological monitoring indicators from the viewer up to the first time point, the dampening factor to be applied to relative motions between the viewer and VR imagery to be rendered to the viewer at the time point no earlier than the first time point, the dampening factor being determined in a predictive model that comprises (1) a dynamics model that relates the viewer's previous queasiness states to the viewer's current queasiness state and (2) an observation model that relates the viewer's physiological monitoring indicators to the viewer's queasiness state;
generating the VR imagery to be rendered to the viewer with the relative motions dampened in accordance with the dampening factor determined in the predictive model.

### Claim 7 (depends on 6)

The method of claim 6, further comprising:
based at least in part on the one or more physiological monitoring indicators from a viewer up to a first time point and one or more queasiness states of the viewer up to the first time point, using the predictive model to predict the viewer's queasiness state at the time point;
determining, based at least in part on the viewer's queasiness state at the time point as predicted, the dampening factor to be applied to the relative motions between the viewer and virtual reality (VR) imagery to be rendered to the viewer at the time point.

### Claim 8 (depends on 6)

The method of claim 6, wherein the predictive model is used to further integrate view-specific initial settings and user input as entered by the viewer up to the first time point into a determination of the dampen factor to be applied to the relative motions between the viewer and virtual reality (VR) imagery to be rendered to the viewer at the time point.

### Claim 9 (depends on 6)

The method of claim 6, wherein the one or more physiological monitoring indicators from the viewer comprise one or more of: eye tracking, pupilometry, galvanic skin response, heart rate, facial expression, thermal imaging, or electroencephalogram.

### Claim 10 (depends on 6)

The method of claim 6, wherein the VR application represents a real time application; wherein the first time point represents a present time that most recently occurs in real time; and wherein the time point represents a future time that has not occurred in real time.

### Claim 11 (depends on 6)

The method of claim 6, wherein the predictive model is implemented with an estimator that estimates or predicts the viewer's future queasiness state and an optimal controller that determines an optimal value for the dampening factor to control the viewer's future queasiness state below a threshold specific to the viewer.

### Claim 12 (depends on 11)

The method of claim 11, wherein the estimator uses one of: a linear state transition function, or a non-linear state transition function.

### Claim 13 (depends on 11)

The method of claim 11, wherein the optimal controller represents a linear quadratic Gaussian controller.

### Claim 14 (depends on 6)

The method of claim 6, wherein the relative motions between the viewer and the VR imagery to be rendered to the viewer at the time point, as dampened by the dampening factor, represent motions at the time point in the viewer's specific motion pathway in a virtual or a represented world.

### Claim 15 (depends on 6)

The method of claim 6, wherein the relative motions between the viewer and the VR imagery to be rendered to the viewer at the time point, as dampened by the dampening factor, comprise one or more of: linear positions, angular positions, translational velocities, rotational velocities, translational accelerations, or rotational accelerations.

### Claim 16 (depends on 6)

The method of claim 6, wherein the dynamics model incorporates one or more previous values of the dampening factor as a part of inputs.

### Claim 17 (depends on 6)

The method of claim 6, wherein the observation model one of: a linear observation model, or a non-linear observation model.

### Claim 18 (depends on 1)

The method of claim 1, wherein the VR application represents one of: a VR cinema application, a VR application incorporating ego-motions, a VR application of high interactivity, a remote-presence application, a game application, or an augmented reality application.

### Claim 19 (depends on 1)

The method of claim 1, further comprising adapting the VR imagery to be rendered to the viewer at the time point to device-specific VR imagery for display to the viewer, wherein the device-specific VR imagery is for rendering with one of: a head-mounted display, an integrated display of a computing device, a display of a wearable computing device, a mobile device, a wall display, or a game display.

### Claim 20 (depends on 1)

The method of claim 1, wherein the VR imagery to be rendered to the viewer at the time point, as dampened by the dampening factor, represents one of: two-dimensional (2D) imagery, three-dimensional (3D) imagery, or multi-view imagery.

### Claim 21 (depends on 1)

An apparatus performing with one or more processors the method of claim 1.

### Claim 22 (depends on 1)

A system with one or more processors performing the method of claim 1.

### Claim 23 (depends on 1)

A non-transitory computer readable storage medium, storing software instructions, which when executed by one or more processors cause performance of the method recited in claim 1.

### Claim 24 (depends on 1)

A computing device comprising one or more processors and one or more storage media, storing a set of instructions, which when executed by one or more processors cause performance of the method recited in claim 1.
