# US 2021/0232632 A1 · Multi-modal virtual experiences of distributed content

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0232632 A1
- Document ID: US20210232632A1
- Status: Published application
- Kind: A1
- Application number: 15/734,664
- Filed: 2018-06-22
- Published: 2021-07-29
- Earliest claimed: 2018-06-22
- Assignees: Virtual Album Technologies LLC
- Applicants: Virtual Album Technologies LLC
- Inventors: Todd A HOWARD
- Web page: https://patentgrep.com/patent/US20210232632A1

## CPC

- G06F 16/335: Filtering based on additional data, e.g. user or group profiles (filtering in web context G06F16/9535, G06F16/9536)
- G06F 16/9035: Filtering based on additional data, e.g. user or group profiles
- G02B 30/52: the three-dimensional [3D] volume being constructed from a stack or sequence of two-dimensional [2D] planes, e.g. depth sampling systems
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 16/211: {Schema design and management}
- G06F 16/27: Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
- G06F 16/9038: Presentation of query results
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/0475: Generative networks
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- H04N 13/395: with depth sampling, i.e. the volume being constructed from a stack or sequence of two-dimensional [2D] image planes
- G06F 2203/011: Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Systems and techniques are described herein for providing a beholder, via a user interface on a user experience device, with a multi-faceted and flexibly-dimensional virtual experience of one or more target identities placed in a subject matter context. A systems aspects include selecting, finding, and interpreting digital content pertaining to a subject matter context indicated by the beholder; deconstructing digital content into discrete content elements containing content segments that are classified according to a schema of element facets, and then constructing a facet-segmented repository of discrete content elements pertaining to a target identity; supplementing existing digital content with supplemental information and content to support the presentation of expanded information content, dimensions, or sensory capabilities; and creating and presenting a virtual experience container that is adapted to the beholder and the capabilities of the beholders user experience device.

## Claims

### Claim 1

A system for constructing a virtual experience of target identities to a user experience device, the system comprising:
computer-readable storage media;
a processing system;
program instructions stored on the computer-readable storage media that, when executed by the processing system, direct the processing system to:
in response to receiving a repository compilation request comprising a designator data structure for a target identity:
from the repository compilation request, determine a set of element sources and query terms for a search of the set of element sources,
send at least one query comprising the query terms to the set of element sources,
receive search results from the search of the set of element sources,
deconstruct the search results into discrete content elements classified in accordance with a schema of element facets,
construct a facet-segmented repository including the discrete content elements, wherein the facet-segmented repository is associated with the designator data structure for the target identity, and
store the facet-segmented repository on the computer-readable storage media.

### Claim 2 (depends on 1)

The system of claim 1, wherein the repository compilation request further comprises a topical limiter.

### Claim 3 (depends on 1)

The system of claim 1, further comprising program instructions that, when executed by the processing system, further direct the processing system to:
in response to receiving, from a user experience device, a beholder request for a virtual experience, wherein the beholder request comprises a subject matter prompt, selected designators of one or more selected target identities, and user experience device parameters:
determine selected discrete content elements, stored in facet-segmented repositories on the computer-readable storage media, that are associated with the selected designators of the one or more selected target identities, and that are aligned with the subject matter prompt,
assemble a virtual experience container from the selected discrete content elements in accordance with the user experience device parameters, and
provide the virtual experience container to the user experience device.

### Claim 4 (depends on 3)

The system of claim 3, wherein the subject matter prompt comprises one or more of:
a descriptor of an actual event;
a description of a memory;
a description of a life experience;
a topic of conversation;
a gesture, wherein the gesture is indicated through a sensor of the user experience device, or wherein the gesture is indicated by a recorded gesture submitted in the beholder request;
a description of a hypothetical situation; and
a fictional event.

### Claim 5 (depends on 3)

The system of claim 3, wherein the program instructions to assemble the virtual experience container comprise program instructions that direct the processing system to:
determine sensory-effect capabilities of the user experience device from the user experience device parameters; and
modify the virtual experience container in accordance with the sensory-effect capabilities of the user experience device.

### Claim 6 (depends on 5)

The system of claim 5, wherein the program instructions to modify the virtual experience container in accordance with the sensory-effect capabilities of the user experience device comprise program instructions that direct the processing system to add a sensory facet retrieved from a sensory element source to the virtual experience container.

### Claim 7 (depends on 5)

The system of claim 5, wherein the program instructions to modify the virtual experience container in accordance with the sensory-effect capabilities of the user experience device comprise program instructions that direct the processing system to expand a dimension of content in the virtual experience container by one or more of:
associating depth elements from a visual element source with two-dimensional visual content elements in the virtual experience container; and
associating temporality progression elements with content elements in the virtual experience container.

### Claim 8 (depends on 3)

The system of claim 3, wherein the program instructions that direct the processing system to assemble the virtual experience container comprise program instructions that direct the processing system to modify the selected discrete content elements in accordance with beholder properties, stored on the computer-readable storage media, of a beholder associated with the beholder request.

### Claim 9 (depends on 3)

The system of claim 3, wherein the program instructions that direct the processing system to determine selected discrete content elements comprise program instructions that direct the processing system to:
use a trained neural network, wherein the trained neural network is trained to associate existing content patterns with various subject matter prompts, to identify selected discrete content elements in the facet-segmented repositories given an input of the subject matter prompt.

### Claim 10 (depends on 1)

The system of claim 1, further comprising program instructions that, when executed by the processing system, further direct the processing system to:
analyze the facet-segmented repository associated with the target identity to determine correlated subject matter prompts from the discrete content elements; and
in response to receiving, from a user experience device, a request for a virtual experience including the target identity, provide at least one suggested subject matter prompt to the user experience device from the correlated subject matter prompts.

### Claim 11 (depends on 10)

The system of claim 10, further comprising program instructions that, when executed by the processing system, further direct the processing system to:
generate a pre-compiled virtual experience container using the discrete content elements related to one or more of the correlated subject matter prompts; and
in response to receiving, from a user experience device, a beholder request for a virtual experience including the target identity and having a subject matter prompt matching a particular correlated subject matter prompt, provide the pre-compiled virtual experience container for the particular correlated subject matter prompt.

### Claim 12 (depends on 1)

The system of claim 1, wherein the schema of element facets comprises one or more of: a primary concept facet type, a place facet type, a temporal facet type, an environment facet type, a person identity facet type, an emotion facet type, a sentiment facet type, a personality trait facet type, a supplemental concept facet type, a sensory facet type, and a cultural facet type.

### Claim 13 (depends on 1)

The system of claim 1, wherein the set of element sources comprises one or more of:
sensor data of the target identity recorded by a third-party device;
content from a chatbot interaction with the target identity;
a digital inventory of possessions of the target identity;
beholder-submitted content relating to the target identity;
a previously-stored facet-segmented repository of the target identity;
a third-party facet-segmented repository;
a genealogy database; and
content from a DNA analysis of the target identity.

### Claim 14 (depends on 1)

The system of claim 1, wherein the program instructions that direct the processing system to construct the facet-segmented repository comprise program instructions that direct the processing system to, iteratively:
analyze the discrete content elements in the facet-segmented repository with reference to the schema of element facets;
extrapolate supplemental information comprising at least one of a supplemental discrete content element, a facet type, and a facet attribute;
search a supplemental set of element sources for the supplemental information; and
modify the facet-segmented repository with the supplemental information.

### Claim 15

A user experience device for selecting and rendering a virtual experience of target identities, the device comprising:
a communications interface;
a processing system;
a user interface system;
a sensory-effect hardware array;
computer-readable storage media;
program instructions on the computer-readable storage media that, when executed by the processing system, direct the processing system to:
upon receiving an indication via the user interface system, send, via the communications interface, a beholder request comprising a subject matter prompt, selected designators of one or more selected identities, and user experience device parameters including a capability manifest of the sensory-effect hardware array;
upon receiving a virtual experience container via the communications interface, render content elements and sensory effects indicated in the virtual experience container on the sensory-effect hardware array.

### Claim 16 (depends on 15)

The user experience device of claim 15, further comprising program instructions that, when executed by the processing system, further direct the processing system to:
render, via the user interface system, a user interface comprising selectable artifacts arrayed in an anchor setting associated with a beholder account,
wherein indicating a selected digital artifact generates the indication and the subject matter prompt for the beholder request.

### Claim 17 (depends on 15)

The user experience device of claim 15, further comprising program instructions that, when executed by the processing system, further direct the processing system to:
render, via the user interface system, a timescale selector for indicating a beholder preference for a timescale of the virtual experience container, wherein indicating the beholder preference for the timescale includes the beholder preference in the beholder request.

### Claim 18 (depends on 15)

The user experience device of claim 15, wherein the sensory-effect hardware array comprises components for sensory expansion including at least one of touch feedback components, gustatory components, and olfactory components.

### Claim 19 (depends on 15)

The user experience device of claim 15, wherein the sensory-effect hardware array comprises components for depicting lesser-dimensional content as higher-dimensional content.

### Claim 20 (depends on 19)

The user experience device of claim 19, wherein the components for depicting lesser-dimensional content as higher-dimensional content comprise a three-dimensional free-space volumetric display.

### Claim 21 (depends on 15)

The user experience device of claim 15, wherein the sensory-effect hardware array comprises virtual reality components.
