Methods and system for generating and displaying 3d videos in a virtual, augmented, or mixed reality environment
AssigneeMagic Leap, Inc.
InventorsPraveen BABU J D, Sean Christopher RILEY, Jeffrey Scott Sommers
Life of this patent
Estimating the term…
Abstract
Disclosed is an approach for displaying 3D videos in a VR and/or AR system. The 3D videos may include 3D animated objects that escape from the display screen. The 3D videos may interact with objects within the VR and/or AR environment. The 3D video may be interactive with a user such that based on user input corresponding to decisions elected by the user at certain portions of the 3D video such that a different storyline and possibly a different conclusion may result for the 3D video. The 3D video may be a 3D icon displayed within a portal of a final 3D render world.
Claims31
A method of displaying 3D video that extends beyond a surface of a display screen, the method comprising:
- identifying a 3D video;
- generating one or more 3D models corresponding to the 3D video;
- and
- rendering the one or more 3D models at an appropriate trigger time along with a rendering of the 3D video.
A display system for displaying 3D video that extends beyond a surface of a display screen, the system comprising:
- an augmented reality head-mounted display system;
- and
- one or more modules for processing data, wherein the one or more modules are stored in one or more memory, the one or more modules configured to perform:
- identifying a 3D video,
- generating one or more 3D models corresponding to the 3D video, and
- rendering the one or more 3D models at an appropriate trigger time along with a rendering of the 3D video.
A video file format comprising:
- one or more animation streams;
- a data store;
- and
- at least one of a video stream or an audio stream.
A method comprising:
- receiving a video file of a video, the video file comprising:
- one or more animation streams;
- a data store comprising control data;
- and
- at least one of a video stream or an audio stream;
- dynamically generating a control stream from the control data and a timeline controller;
- loading a model of a 3D object received from the data store;
- and
- attaching the 3D object to an animation stream of the one or more animation streams.
A computer system implementing a mixed reality video player, comprising:
- a computer processor to execute a set of program code instructions;
- and
- a memory to hold the program code instructions, in which the program code instructions comprises program code to perform:
- receiving a video file of a video, the video file comprising:
- one or more animation streams,
- a data store comprising control data, and
- at least one of a video stream or an audio stream,
- dynamically generating a control stream from the control data and a timeline controller,
- loading a model of a 3D object received from the data store, and
- attaching the 3D object to an animation stream of the one or more animation streams.
A method comprising:
- receiving a video file of a video, the video file comprising
- one or more animation streams;
- a data store comprising control data;
- and
- at least one of a video stream or an audio stream;
- dynamically generating a control stream from the control data and a timeline controller;
- requesting a user interaction answering a question displayed to the user at a point in time of the video;
- receiving an answer to the question via the user interaction;
- notifying a timeline controller of the answer;
- skipping to an appropriate portion of the video corresponding to the answer;
- and
- displaying the video from the appropriate portion.
A computer system implementing a mixed reality video player, comprising:
- a computer processor to execute a set of program code instructions;
- and
- a memory to hold the program code instructions, in which the program code instructions comprises program code to perform:
- receiving a video file of a video, the video file comprising
- one or more animation streams;
- a data store comprising control data;
- and
- at least one of a video stream or an audio stream;
- dynamically generating a control stream from the control data and a timeline controller;
- requesting a user interaction answering a question displayed to the user at a point in time of the video;
- receiving an answer to the question via the user interaction;
- notifying a timeline controller of the answer;
- skipping to an appropriate portion of the video corresponding to the answer;
- and
- displaying the video from the appropriate portion.
A method of displaying an environment aware 3D video in a virtual and/or augmented reality environment, the method comprising:
- identifying a 3D video;
- receiving, from one or more sensors, environment information of a user environment, the environment information identifying objects within the environment;
- and
- rendering a portion of the 3D video onto one or more objects identified from the environment.
A virtual and/or augmented reality display system comprising:
- a camera to capture a virtual and/or augmented reality environment;
- and
- a module for processing data, wherein the module is stored in a memory, the module when executed, performs a process of:
- identifying a 3D video,
- receiving, from one or more sensors, environment information of a user environment, the environment information identifying objects within the environment, and
- rendering a portion of the 3D video onto one or more objects identified from the environment.
A method of displaying interactive 3D video in a virtual and/or augmented reality environment, the method comprising:
- identifying a 3D video as a stream of non-executable data periodically interrupted by decision nodes;
- receiving one or more interactions from a user as input corresponding to decisions elected by the user;
- and
- rendering the 3D video based at least on the decisions elected by the user.
A virtual and/or augmented reality display system comprising:
- a camera to capture a virtual and/or augmented reality environment;
- and
- a module for processing data, wherein the module is stored in a memory, the module when executed, performs a process of:
- identifying a 3D video as a stream of non-executable data periodically interrupted by decision nodes,
- receiving one or more interactions from a user as input corresponding to decisions elected by the user, and
- rendering the 3D video based at least on the decisions elected by the user.
Dependencies are inferred from the claim text.