# US 2019/0156565 A1 · Methods and Systems for Distinguishing Objects in a Natural Setting to Create an Individually-Manipulable Volumetric Model of an Object

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0156565 A1
- Document ID: US20190156565A1
- Status: Published application
- Kind: A1
- Application number: 16/234,448
- Filed: 2018-12-27
- Published: 2019-05-23
- Earliest claimed: 2016-04-28
- Assignees: Verizon Patent and Licensing Inc.
- Applicants: Verizon Patent and Licensing Inc.
- Inventors: Mohammad Raheel Khalid; Ali Jaafar; Denny Breitenfeld; Xavier Hansen; Christian Egeler; Syed Kamal; Lama Hewage Ravi Prathapa Chandrasiri; Steven L. Smith
- Web page: https://patentgrep.com/patent/US20190156565A1

## CPC

- G06T 17/00: Three-dimensional [3D] modelling for computer graphics
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

An exemplary virtual reality media provider system receives two-dimensional (“2D”) video data for surfaces of first and second objects located in a natural setting. The 2D video data is captured by first and second capture devices disposed at different positions with respect to the objects. The system distinguishes the first object from the second object by performing a plurality of techniques in combination with one another. The plurality of techniques include determining that the first object is moving in relation to the second object; and determining that, from a vantage point of at least one of the different positions, a representation of the first object captured within the 2D video data does not overlap with a representation of the second object. Based on the received 2D video data and the distinguishing of the first and second objects, the system generates an individually-manipulable volumetric model of the first object.

## Claims

### Claim 1

A method comprising:
receiving, by a virtual reality media provider system, two-dimensional (“2D”) video data for surfaces of a first object and a second object located in a natural setting, the 2D video data captured by a first capture device and a second capture device disposed at different positions with respect to the first and second objects;
distinguishing, by the virtual reality media provider system, the first object from the second object by performing a plurality of techniques, the plurality of techniques including:
determining, based on the received 2D video data, that the first object is moving in relation to the second object, and
determining, based on the received 2D video data, that, from a vantage point of at least one of the different positions at which the first and second capture devices are disposed, a representation of the first object captured within the 2D video data does not overlap with a representation of the second object; and
generating, by the virtual reality media provider system based on the received 2D video data and the distinguishing of the first object from the second object, an individually-manipulable volumetric model of the first object.

### Claim 2 (depends on 1)

The method of claim 1, further comprising receiving, by the virtual reality media provider system along with the 2D video data, depth data for the surfaces of the first and second objects;
wherein the first and second capture devices are implemented as synchronous video and depth capture devices.

### Claim 3 (depends on 2)

The method of claim 2, wherein the plurality of techniques further includes determining, based on the received depth data, that the first object is located at least a predetermined distance away from the second object.

### Claim 4 (depends on 1)

The method of claim 1, wherein the plurality of techniques further includes:
identifying, based on the received 2D video data, the first object and the second object; and
determining, based on the identifying of the first and second objects, that the first object is different from the second object.

### Claim 5 (depends on 1)

The method of claim 1, wherein:
the natural setting is a setting of a real-world event;
the different positions with respect to the first and second objects where the first and second capture devices are disposed include a plurality of fixed positions surrounding the first and second objects within the setting of the real-world event.

### Claim 6 (depends on 1)

The method of claim 1, wherein:
the individually-manipulable volumetric model of the first object is configured to be individually manipulated with respect to an immersive virtual reality world while a user of a media player device is experiencing the immersive virtual reality world using the media player device; and
the immersive virtual reality world is based on virtual reality media content provided to the media player device and representative of the immersive virtual reality world.

### Claim 7 (depends on 6)

The method of claim 6, wherein the individually-manipulable volumetric model of the first object is configured to be individually manipulated with respect to the immersive virtual reality world by inserting the individually-manipulable volumetric model of the first object into the immersive virtual reality world at a particular location within the immersive virtual reality world.

### Claim 8 (depends on 6)

The method of claim 6, wherein the individually-manipulable volumetric model of the first object is configured to be individually manipulated with respect to the immersive virtual reality world by replacing the individually-manipulable volumetric model of the first object included within the immersive virtual reality world with an individually-manipulable volumetric model of a third object within the immersive virtual reality world.

### Claim 9 (depends on 6)

The method of claim 6, wherein the individually-manipulable volumetric model of the first object is configured to be individually manipulated with respect to the immersive virtual reality world by removing the individually-manipulable volumetric model of the first object from the immersive virtual reality world.

### Claim 10

A system comprising:
a first capture device disposed at a first position with respect to a first object and a second object located in a natural setting, the first capture device configured to capture two-dimensional (“2D”) video data for surfaces of the first and second objects from the first position;
a second capture device disposed at a second position with respect to the first and second objects, the second capture device configured to further capture the 2D video data for the surfaces of the first and second objects from the second position;
a memory storing instructions; and
a processor communicatively coupled to the memory and to the first and second capture devices, the processor configured to execute the instructions to:
receive, from the first and second capture devices, the 2D video data for the surfaces of the first and second objects;
distinguish the first object from the second object by performing a plurality of techniques, the plurality of techniques including:
determining, based on the received 2D video data, that the first object is moving in relation to the second object, and
determining, based on the received 2D video data, that, from a vantage point of at least one of the first and second positions, a representation of the first object captured within the 2D video data does not overlap with a representation of the second object; and
generate, based on the received 2D video data and the distinguishing of the first object from the second object, an individually-manipulable volumetric model of the first object.

### Claim 11 (depends on 10)

The system of claim 10, wherein:
the processor is further configured to execute the instructions to receive, from the first and second capture devices along with the 2D video data, depth data for the surfaces of the first and second objects;
the first and second capture devices are implemented as synchronous video and depth capture devices.

### Claim 12 (depends on 11)

The system of claim 11, wherein the plurality of techniques further includes:
determining, based on the received depth data, that the first object is located at least a predetermined distance away from the second object; and
identifying, based on the received 2D video data, the first object and the second object and determining that the first object is different from the second object.

### Claim 13 (depends on 10)

The system of claim 10, wherein:
the natural setting is a setting of a real-world event;
the different positions with respect to the first and second objects where the first and second capture devices are disposed include a plurality of fixed positions surrounding the first and second objects within the setting of the real-world event.

### Claim 14 (depends on 10)

The system of claim 10, wherein:
the individually-manipulable volumetric model of the first object is configured to be individually manipulated with respect to an immersive virtual reality world while a user of a media player device is experiencing the immersive virtual reality world using the media player device; and
the immersive virtual reality world is based on virtual reality media content provided to the media player device and representative of the immersive virtual reality world.

### Claim 15 (depends on 14)

The system of claim 14, wherein the individually-manipulable volumetric model of the first object is configured to be individually manipulated with respect to the immersive virtual reality world by at least one of:
inserting the individually-manipulable volumetric model of the first object into the immersive virtual reality world at a particular location within the immersive virtual reality world;
replacing the individually-manipulable volumetric model of the first object included within the immersive virtual reality world with an individually-manipulable volumetric model of a third object within the immersive virtual reality world; and
removing the individually-manipulable volumetric model of the first object from the immersive virtual reality world.

### Claim 16

A non-transitory computer-readable medium storing instructions that, when executed, direct a processor of a computing device to:
receive two-dimensional (“2D”) video data for surfaces of a first object and a second object located in a natural setting, the 2D video data captured by a first capture device and a second capture device disposed at different positions with respect to the first and second objects;
distinguish the first object from the second object by performing a plurality of techniques, the plurality of techniques including:
determining, based on the received 2D video data, that the first object is moving in relation to the second object, and
determining, based on the received 2D video data, that, from a vantage point of at least one of the different positions at which the first and second capture devices are disposed, a representation of the first object captured within the 2D video data does not overlap with a representation of the second object; and
generate, based on the received 2D video data and the distinguishing of the first object from the second object, an individually-manipulable volumetric model of the first object.

### Claim 17 (depends on 16)

The non-transitory computer-readable medium of claim 16, wherein:
the instructions further direct the processor to receive, along with the 2D video data, depth data for the surfaces of the first and second objects; and
the first and second capture devices are implemented as synchronous video and depth capture devices.

### Claim 18 (depends on 17)

The non-transitory computer-readable medium of claim 17, wherein the plurality of techniques further includes:
determining, based on the received depth data, that the first object is located at least a predetermined distance away from the second object; and
identifying, based on the received 2D video data, the first object and the second object and thereby determining that the first object is different from the second object.

### Claim 19 (depends on 16)

The non-transitory computer-readable medium of claim 16, wherein:
the natural setting is a setting of a real-world event;
the different positions with respect to the first and second objects where the first and second capture devices are disposed include a plurality of fixed positions surrounding the first and second objects within the setting of the real-world event.

### Claim 20 (depends on 16)

The non-transitory computer-readable medium of claim 16, wherein:
the individually-manipulable volumetric model of the first object is configured to be individually manipulated with respect to an immersive virtual reality world while a user of a media player device is experiencing the immersive virtual reality world using the media player device; and
the immersive virtual reality world is based on virtual reality media content provided to the media player device and representative of the immersive virtual reality world.
