# US 10,373,342 B1 · Content generation in an immersive environment > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,373,342 B1 - Document ID: US10373342B1 - Status: Granted - Kind: B1 - Application number: 15/730,476 - Filed: 2017-10-11 - Granted: 2019-08-06 - Earliest claimed: 2017-01-10 - Assignees: Lucasfilm Entertainment Company Ltd. - Applicants: Lucasfilm Entertainment Company Ltd. - Inventors: Jose Perez, III; Peter Dollar; Barak Moshe - Web page: https://patentgrep.com/patent/US10373342B1 ## CPC - G06T 7/97: {Determining parameters from multiple pictures (depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85)} - G02B 27/017: {Head mounted} - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06F 3/0346: with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors - G06F 3/04815: Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object - G06F 3/0482: Interaction with lists of selectable items, e.g. menus - G06F 3/0484: for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range - G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) - G06F 3/04845: for image manipulation, e.g. dragging, rotation, expansion or change of colour - G06F 3/04847: Interaction techniques to control parameter settings, e.g. interaction with sliders or dials - G06F 3/0488: using a touch-screen or digitiser, e.g. input of commands through traced gestures - G06T 7/73: using feature-based methods - G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts - H04L 67/104: Peer-to-peer [P2P] networks - H04L 67/125: involving control of end-device applications over a network - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract Views of a virtual environment can be displayed on mobile devices in a real-world environment simultaneously for multiple users. The users can operate selections devices in the real-world environment that interact with objects in the virtual environment. Virtual characters and objects can be moved and manipulated using selection shapes. A graphical interface can be instantiated and rendered as part of the virtual environment. Virtual cameras and screens can also be instantiated to created storyboards, backdrops, and animated sequences of the virtual environment. These immersive experiences with the virtual environment can be used to generate content for users and for feature films. ## Claims ### Claim 1 A method comprising: causing a virtual environment to be displayed on a display of a virtual-reality headset in a real-world environment; receiving first inputs from a selection device to use one or more virtual cameras in the virtual environment; capturing a plurality of images from the one or more virtual cameras in the virtual environment; displaying the plurality of images on a virtual storyboard display in the virtual environment; receiving second inputs from the selection device to select or reorder the plurality of the images on the virtual storyboard display in the virtual environment such that the plurality of images on the virtual storyboard define a scene that is based on the virtual environment; and generating a video sequence based on the plurality of images. ### Claim 2 (depends on 1) The method of claim 1, wherein the one or more virtual cameras comprise a plurality of virtual cameras at different locations in the virtual environment. ### Claim 3 (depends on 2) The method of claim 2, wherein each of the plurality of virtual cameras are different from virtual cameras that capture a view displayed on the virtual-reality headset. ### Claim 4 (depends on 1) The method of claim 1, further comprising adding a virtual screen as a rendered object in the virtual environment. ### Claim 5 (depends on 4) The method of claim 4, wherein the virtual screen displays a view captured from the one or more virtual cameras in real-time. ### Claim 6 (depends on 4) The method of claim 4, wherein the virtual screen displays a static image that is loaded from outside of the virtual environment. ### Claim 7 (depends on 4) The method of claim 4, wherein the virtual screen displays a video sequence that is loaded from outside of the virtual environment. ### Claim 8 A system comprising: one or more processors; and one or more memories comprising instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising: causing a virtual environment to be displayed on a display of a virtual-reality headset in a real-world environment; receiving first inputs from a selection device to use one or more virtual cameras in the virtual environment; capturing a plurality of images from the one or more virtual cameras in the virtual environment; displaying the plurality of images on a virtual storyboard display in the virtual environment; receiving second inputs from the selection device to select or reorder the plurality of the images on the virtual storyboard display in the virtual environment such that the plurality of images on the virtual storyboard define a scene that is based on the virtual environment; and generating a video sequence based on the plurality of images. ### Claim 9 (depends on 8) The system of claim 8, wherein the inputs to use the one or more virtual cameras in the virtual environment comprise inputs that cause change a focal length of at least one of the one or more virtual cameras. ### Claim 10 (depends on 8) The system of claim 8, wherein at least one of the one or more virtual cameras is displayed in the virtual environment as a screen that plays a view captured by the at least one of the one or more virtual cameras. ### Claim 11 (depends on 8) The system of claim 8, wherein the virtual environment is a model of a real-world set. ### Claim 12 (depends on 11) The system of claim 11, wherein the virtual environment is generated by scanning the real-world set. ### Claim 13 (depends on 11) The system of claim 11, wherein generating the video sequence based on the plurality of images comprises filming a scene in the real-world set. ### Claim 14 (depends on 8) The system of claim 8, wherein generating the video sequence based on the plurality of images comprises rendering a scene from the virtual environment. ### Claim 15 (depends on 8) The system of claim 8, wherein the operations further comprise receiving an input that adjusts a sunlight source in the virtual environment. ### Claim 16 (depends on 15) The system of claim 15, wherein adjusting the sunlight source comprises adjusting a position of the sunlight source in a sky of the virtual environment. ### Claim 17 A non-transitory computer-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising: causing a virtual environment to be displayed on a display of a virtual-reality headset in a real-world environment; receiving first inputs from a selection device to use one or more virtual cameras in the virtual environment; capturing a plurality of images from the one or more virtual cameras in the virtual environment; displaying the plurality of images on a virtual storyboard display in the virtual environment; receiving second inputs from the selection device to select or reorder the plurality of the images on the virtual storyboard display in the virtual environment such that the plurality of images on the virtual storyboard define a scene that is based on the virtual environment; and generating a video sequence based on the plurality of images. ### Claim 18 (depends on 17) The non-transitory computer-readable medium of claim 17, wherein the one or more virtual cameras comprise a plurality of virtual cameras at different locations in the virtual environment, and wherein each of the plurality of virtual cameras are different from virtual cameras that capture a view displayed on the virtual-reality headset. ### Claim 19 (depends on 17) The non-transitory computer-readable medium of claim 17, wherein the operations further comprise adding a virtual screen as a rendered object in the virtual environment, wherein the virtual screen displays a view captured from the one or more virtual cameras in real-time. ### Claim 20 (depends on 17) The non-transitory computer-readable medium of claim 17, wherein at least one of the one or more virtual cameras is displayed in the virtual environment as a screen that plays a view captured by the at least one of the one or more virtual cameras.