# US 2019/0172496 A1 · Methods and Systems for Customizing Virtual Reality Data

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0172496 A1
- Document ID: US20190172496A1
- Status: Published application
- Kind: A1
- Application number: 16/267,227
- Filed: 2019-02-04
- Published: 2019-06-06
- Earliest claimed: 2017-05-31
- Assignees: Verizon Patent and Licensing Inc.
- Applicants: Verizon Patent and Licensing Inc.
- Inventors: Michael Lodato; Pai Moodlagiri; Rene Sepulveda
- Web page: https://patentgrep.com/patent/US20190172496A1

## CPC

- G11B 27/036: Insert-editing
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G11B 27/031: Electronic editing of digitised analogue information signals, e.g. audio or video signals
- G11B 27/10: Indexing; Addressing; Timing or synchronising; Measuring tape travel
- G11B 27/32: on separate auxiliary tracks of the same or an auxiliary record carrier
- H04N 5/9201: {involving the multiplexing of an additional signal and the video signal}
- H04N 9/8205: {involving the multiplexing of an additional signal and the colour video signal}
- H04N 13/161: Encoding, multiplexing or demultiplexing different image signal components (for multi-view video sequence encoding H04N19/597)
- H04N 13/189: Recording image signals; Reproducing recorded image signals
- H04N 13/204: using stereoscopic image cameras (stereoscopic photography G03B35/00)
- H04N 13/366: using viewer tracking
- H04N 19/597: specially adapted for multi-view video sequence encoding
- H04N 21/21805: {enabling multiple viewpoints, e.g. using a plurality of cameras}
- H04N 21/23605: {Creation or processing of packetized elementary streams [PES]}
- H04N 21/23614: {Multiplexing of additional data and video streams (arrangements for simultaneous broadcast of plural pieces of information H04H20/28)}
- H04N 21/47217: {for controlling playback functions for recorded or on-demand content, e.g. using progress bars, mode or play-point indicators or bookmarks (specific graphical features in visual interfaces H04N21/4312)}
- H04N 21/8146: {involving graphical data, e.g. 3D object, 2D graphics}
- H04N 21/816: {involving special video data, e.g 3D video}
- H04N 13/257: Colour aspects
- H04N 13/324: Colour aspects
- G11B: INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER (recording measured values in a way that does not require playback through a transducer G01D9/00; recording or playback apparatus using mechanically marked tape, e.g. punched paper tape, or using unit records, e.g. punched or magnetically marked cards G06K; transferring data from one type of record carrier to another G06K1/18; circuits for coupling output of reproducer to radio receiver H04B1/20; gramophone pick-ups or like acoustic electromechanical transducers or circuits therefor H04R)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

An exemplary virtual reality system accesses metadata describing a plurality of frame sequences that each include two-dimensional color data frames and depth data frames that depict a different view of a three-dimensional (3D) scene. The virtual reality system identifies a set of experience parameters defining a particular virtual reality experience associated with the 3D scene and that is to be experienced by a user using a media player device. Based on the metadata and the set of experience parameters, the virtual reality system selects frame sequences from the plurality of frame sequences for inclusion in a frame sequence subset that is customized to the particular virtual reality experience and that collectively includes data sufficient to allow the media player device to generate a virtual 3D representation of the 3D scene. The virtual reality system also provides the selected frame sequences of the frame sequence subset to the media player device.

## Claims

### Claim 1

A method comprising:
accessing, by a virtual reality system, metadata descriptive of a plurality of frame sequences that each include two-dimensional (2D) color data frames and depth data frames that depict a different view of a three-dimensional (3D) scene;
identifying, by the virtual reality system, a set of experience parameters defining a particular virtual reality experience associated with the 3D scene and that is to be experienced by a user using a media player device;
selecting, by the virtual reality system based on the metadata and the set of experience parameters, frame sequences from the plurality of frame sequences for inclusion in a frame sequence subset that is customized to the particular virtual reality experience and that collectively includes data sufficient to allow the media player device to generate a virtual 3D representation of the 3D scene; and
providing, by the virtual reality system to the media player device, the selected frame sequences included within the frame sequence subset.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
accessing, by the virtual reality system, the frame sequences selected for inclusion in the frame sequence subset; and
generating, by the virtual reality system based on the accessed frame sequences selected for inclusion in the frame sequence subset, a virtual reality dataset associated with the frame sequence subset customized to the particular virtual reality experience;
wherein the providing of the selected frame sequences included within the frame sequence subset comprises transmitting the virtual reality dataset to the media player device.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
including, by the virtual reality system based on the selecting of the frame sequences for inclusion in the frame sequence subset, an entry corresponding to the particular virtual reality experience within an experience selection data structure;
wherein:
the experience selection data structure is configured to facilitate the media player device in dynamically selecting different entries, from a plurality of entries included within the experience selection data structure, as the media player device provides different virtual reality experiences to the user, and
the different entries in the plurality of entries correspond to different frame sequence subsets customized to different virtual reality experiences provided by the media player device to the user.

### Claim 4 (depends on 3)

The method of claim 3, further comprising:
providing, by the virtual reality system to the media player device, the experience selection data structure;
receiving, by the virtual reality system as facilitated by the experience selection data structure, a request from the media player device to access the frame sequence subset that is customized to the particular virtual reality experience; and
providing, by the virtual reality system to the media player device in response to the receiving of the request, the frame sequence subset that is customized to the particular virtual reality experience.

### Claim 5 (depends on 1)

The method of claim 1, wherein the identified set of experience parameters defining the particular virtual reality experience includes an experience parameter representative of a position within the 3D scene, the position associated with a vantage point from which the particular virtual reality experience is provided to the user.

### Claim 6 (depends on 5)

The method of claim 5, wherein the frame sequences selected for inclusion in the frame sequence subset that is customized to the particular virtual reality experience collectively surround, along at least two dimensions, the position within the 3D scene that is associated with the vantage point from which the particular virtual reality experience is provided to the user.

### Claim 7 (depends on 1)

The method of claim 1, wherein:
the identifying of the set of experience parameters defining the particular virtual reality experience is performed dynamically such that the particular virtual reality experience is identified as a non-predetermined virtual reality experience; and
the selecting of the frame sequences for inclusion in the frame sequence subset that is customized to the non-predetermined virtual reality experience is performed dynamically in response to the dynamic identifying of the set of experience parameters.

### Claim 8 (depends on 1)

The method of claim 1, wherein:
the 3D scene includes elements of a real-world scene captured by a plurality of capture devices disposed at different locations with respect to the real-world scene so as to capture different views of the real-world scene; and
the plurality of frame sequences includes a virtualized frame sequence that depicts a view of the real-world scene that is unaligned with the different views of the real-world scene captured by the plurality of capture devices.

### Claim 9 (depends on 8)

The method of claim 8, wherein:
the identifying of the set of experience parameters defining the particular virtual reality experience is performed dynamically such that the particular virtual reality experience is identified as a non-predetermined virtual reality experience;
the method further comprises generating, by the virtual reality system in response to the dynamic identifying of the set of experience parameters, the virtualized frame sequence that depicts the view of the real-world scene that is unaligned with the different views captured by the plurality of capture devices; and
the frame sequence subset that is customized to the non-predetermined virtual reality experience includes the virtualized frame sequence.

### Claim 10

A system comprising:
a memory storing instructions; and
a processor communicatively coupled to the memory and configured to execute the instructions to:
access metadata descriptive of a plurality of frame sequences that each include two-dimensional (2D) color data frames and depth data frames that depict a different view of a three-dimensional (3D) scene,
identify a set of experience parameters defining a particular virtual reality experience associated with the 3D scene and that is to be experienced by a user using a media player device,
select, based on the metadata and the set of experience parameters, frame sequences from the plurality of frame sequences for inclusion in a frame sequence subset that is customized to the particular virtual reality experience and that collectively includes data sufficient to allow the media player device to generate a virtual 3D representation of the 3D scene, and
provide the selected frame sequences included within the frame sequence subset to the media player device.

### Claim 11 (depends on 10)

The system of claim 10, wherein:
the processor is further configured to execute the instructions to
access the frame sequences selected for inclusion in the frame sequence subset, and
generate, based on the accessed frame sequences selected for inclusion in the frame sequence subset, a virtual reality dataset associated with the frame sequence subset customized to the particular virtual reality experience; and
the providing of the selected frame sequences included within the frame sequence subset comprises transmitting the virtual reality dataset to the media player device.

### Claim 12 (depends on 10)

The system of claim 10, wherein:
the processor is further configured to execute the instructions to include, based on the selecting of the frame sequences for inclusion in the frame sequence subset, an entry corresponding to the particular virtual reality experience within an experience selection data structure;
the experience selection data structure is configured to facilitate the media player device in dynamically selecting different entries, from a plurality of entries included within the experience selection data structure, as the media player device provides different virtual reality experiences to the user; and
the different entries in the plurality of entries correspond to different frame sequence subsets customized to the different virtual reality experiences provided by the media player device to the user.

### Claim 13 (depends on 12)

The system of claim 12, wherein the processor is further configured to execute the instructions to:
provide the experience selection data structure to the media player device;
receive, as facilitated by the experience selection data structure, a request from the media player device to access the frame sequence subset that is customized to the particular virtual reality experience; and
provide, to the media player device in response to the receiving of the request, the frame sequence subset that is customized to the particular virtual reality experience.

### Claim 14 (depends on 10)

The system of claim 10, wherein the identified set of experience parameters defining the particular virtual reality experience includes an experience parameter representative of a position within the 3D scene, the position associated with a vantage point from which the particular virtual reality experience is provided to the user.

### Claim 15 (depends on 14)

The system of claim 14, wherein the frame sequences selected for inclusion in the frame sequence subset that is customized to the particular virtual reality experience collectively surround, along at least two dimensions, the position within the 3D scene that is associated with the vantage point from which the particular virtual reality experience is provided to the user.

### Claim 16 (depends on 10)

The system of claim 10, wherein:
the identifying of the set of experience parameters defining the particular virtual reality experience is performed dynamically such that the particular virtual reality experience is identified as a non-predetermined virtual reality experience; and
the selecting of the frame sequences for inclusion in the frame sequence subset that is customized to the non-predetermined virtual reality experience is performed dynamically in response to the dynamic identifying of the set of experience parameters.

### Claim 17 (depends on 10)

The system of claim 10, wherein:
the 3D scene includes elements of a real-world scene captured by a plurality of capture devices disposed at different locations with respect to the real-world scene so as to capture different views of the real-world scene; and
the plurality of frame sequences includes a virtualized frame sequence that depicts a view of the real-world scene that is unaligned with the different views of the real-world scene captured by the plurality of capture devices.

### Claim 18 (depends on 17)

The system of claim 17, wherein:
the identifying of the set of experience parameters defining the particular virtual reality experience is performed dynamically such that the particular virtual reality experience is identified as a non-predetermined virtual reality experience;
the method further comprises generating, by the virtual reality system in response to the dynamic identifying of the set of experience parameters, the virtualized frame sequence that depicts the view of the real-world scene that is unaligned with the different views captured by the plurality of capture devices; and
the frame sequence subset that is customized to the non-predetermined virtual reality experience includes the virtualized frame sequence.

### Claim 19

A non-transitory computer-readable medium storing instructions that, when executed, direct a processor of a computing device to:
access metadata descriptive of a plurality of frame sequences that each include two-dimensional (2D) color data frames and depth data frames that depict a different view of a three-dimensional (3D) scene;
identify a set of experience parameters defining a particular virtual reality experience associated with the 3D scene and that is to be experienced by a user using a media player device;
select, based on the metadata and the set of experience parameters, frame sequences from the plurality of frame sequences for inclusion in a frame sequence subset that is customized to the particular virtual reality experience and that collectively includes data sufficient to allow the media player device to generate a virtual 3D representation of the 3D scene; and
provide the selected frame sequences included within the frame sequence subset to the media player device.

### Claim 20 (depends on 19)

The non-transitory computer-readable medium of claim 19, wherein:
the instructions further direct the processor to
access the frame sequences selected for inclusion in the frame sequence subset, and
generate, based on the accessed frame sequences selected for inclusion in the frame sequence subset, a virtual reality dataset associated with the frame sequence subset customized to the particular virtual reality experience; and
the providing of the selected frame sequences included within the frame sequence subset comprises transmitting the virtual reality dataset to the media player device.
