Ad-hoc dynamic capture of an immersive virtual reality experience

A computer-implemented method for dynamically capturing a streamed video experience, comprising:

• displaying a virtual environment onto walls or screens of a surround video projection environment;
• using cameras situated behind the walls or screens to view through perforations, capturing video of an interior space of the surround video projection environment defined by the walls or screens while the virtual environment being display;
• determining, from volumetric capture data of each of one or more elements located in the interior space of the surround video projection environment, one or more potential elements;
• removing the displayed virtual environment from each of the one or more potential elements to provide edge detection to generate one or more detected elements;
• removing projections of the displayed virtual environment from each detected element to generate a captured element corresponding to each detected element;
• and
• combining the displayed virtual environment with the captured elements in a video stream and forwarding the video stream for separate consumption.

A non-transitory computer-readable storage medium including instructions that control one or more computer processors to execute a method comprising:

• displaying a virtual environment onto walls or screens of a surround video projection environment;
• using cameras situated behind the walls or screens to view through perforations, capturing video of an interior space of the surround video projection environment defined by the walls or screens while the virtual environment being displayed;
• determining, from volumetric capture data of each of one or more elements located in the interior space of the surround video projection environment, one or more potential elements;
• removing the display virtual environment from each of the one or more potential elements to provide edge detection to generate one or more detected elements;
• removing projections of the displayed virtual environment from each detected element to generate a captured element corresponding to each detected element;
• and
• combining the displayed virtual environment with the captured elements in a video stream and forwarding the video stream for separate consumption.

A non-transitory computer-readable storage medium including instructions that control one or more computer processors to perform a method comprising:

• receiving streamed video data in substantially real time from one or more video cameras located behind a display surface in a video projection environment while the environment is currently displaying a 360 degree virtual environment, wherein the 360 degree virtual environment defines an interior space of the video projection environment;
• determining volumetric capture data of one or more elements located in the interior space of the video projection environment;
• isolating the one or more elements by removing the displayed virtual environment to provide edge detection to isolate each element from the displayed virtual environment;
• removing projections of the displayed virtual environment onto the one or more isolated elements from each isolated element to generate a captured element;
• generating a composited video stream that includes the captured one or more elements composited with the displayed virtual environment;
• and
• storing the generated composited video stream;
• wherein the stored composited video stream is configured to be streamed by an external virtual reality device.