# US 12,008,771 B2 · Device and method of creating an augmented interactive virtual reality system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,008,771 B2
- Document ID: US12008771B2
- Status: Granted
- Kind: B2
- Application number: 18/117,406
- Filed: 2023-03-04
- Published: 2023-11-02
- Granted: 2024-06-11
- Earliest claimed: 2015-08-28
- Assignees: Vr Drive Sp. Z O.O.
- Applicants: VR Drive sp. z o.o.
- Inventors: Arkadiusz Rogozinski; Radoslaw Gezella; Pawel Tryzno; Piotr Kozinski
- Web page: https://patentgrep.com/patent/US12008771B2

## CPC

- G06T 7/246: using feature-based methods, e.g. the tracking of corners or segments
- G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51)
- G06T 15/08: Volume rendering
- G06T 19/00: Manipulating three-dimensional [3D] models or images for computer graphics
- G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI]
- G06V 20/20: in augmented reality scenes
- G06V 20/41: {Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items (segmenting video sequences G06V20/49)}
- H04N 13/111: Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
- H04N 13/161: Encoding, multiplexing or demultiplexing different image signal components (for multi-view video sequence encoding H04N19/597)
- G06T 2200/24: involving graphical user interfaces [GUIs]
- G06T 2207/10016: Video; Image sequence
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2207/20101: Interactive definition of point of interest, landmark or seed
- G06T 2207/20104: Interactive definition of region of interest [ROI]
- G06T 2207/30241: Trajectory
- G06V 10/945: {User interactive design; Environments; Toolboxes}
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A system for detecting and incorporating three-dimensional objects into a video stream reads an input video data stream. The user specifies areas of attention wherein said areas of attention or hotspots. Tracking movement of the hotspots generating a trajectory of said at least one object. Generating a cloud of points and tracking said points to detect configurations of points most similar to the initially defined hotspot. Obtaining a three dimensional topology defining a volume of interest in a three-dimensional space. Building virtual structures or pseudo objects that are placed within a spherical environment generated on the input video.

## Claims

### Claim 1

A method for adding data into a video stream comprising:
reading an input video data stream;
accepting hotspots defining at least one object of interest shown within the video data stream;
tracking movement of said hotspots generating a trajectory of said at least one object of interest in two-dimensional space of the video;
defining a three-dimensional topology defining a three-dimensional volume of interest in a three-dimensional space;
translating the hotspots to the three-dimensional volume;
calculating motion of the hotspots in three-dimensional space;
building virtual structures to relate said hotspots to said three-dimensional space to create a three dimensional geometric shape; and
projecting resulting said shape.

### Claim 2 (depends on 1)

The method of claim 1 wherein said projecting comprises projecting on a spherical environment.

### Claim 3 (depends on 1)

The method of claim 1 wherein said building virtual structures comprises compiling pseudo objects wherein each pseudo object is defined as a group of hotspots.

### Claim 4 (depends on 1)

The method of claim 1 wherein said video data stream comprises encoded video.

### Claim 5 (depends on 1)

The method of claim 1 wherein said video data stream comprises decoded video stream.

### Claim 6 (depends on 3)

The method of claim 3 wherein additional information is added to said pseudo objects.

### Claim 7 (depends on 6)

The method of claim 6 wherein a spherical projection of video and pseudo objects is presented to a user.

### Claim 8 (depends on 1)

The method of claim 1 wherein said video data stream comprises encoded video.

### Claim 9 (depends on 1)

The method of claim 1 wherein said video data stream comprises decoded video stream.

### Claim 10 (depends on 1)

The method of claim 1 wherein additional information is associated with said at least one created object of interest shown within the video data stream.

### Claim 11 (depends on 10)

The method of claim 10 wherein a spherical projection of video and pseudo objects is presented to a user.

### Claim 12 (depends on 1)

The method of claim 1 wherein said accepting of hotspot input comprises accepting input from a user.

### Claim 13 (depends on 1)

The method of claim 1 wherein said object of interest comprises a real-world object.

### Claim 14 (depends on 1)

The method of claim 1 wherein said defining of a three-dimensional space of interest occurs from user input.