# US 10,121,284 B2 · Virtual camera control using motion control systems for augmented three dimensional reality > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,121,284 B2 - Document ID: US10121284B2 - Status: Granted - Kind: B2 - Application number: 14/495,341 - Filed: 2014-09-24 - Published: 2015-01-08 - Granted: 2018-11-06 - Earliest claimed: 2010-09-01 - Assignees: Disney Enterprises, Inc. - Applicants: Disney Enterprises, Inc. - Inventors: Aaron Thiel; Michael Gay; Anthony Bailey - Web page: https://patentgrep.com/patent/US10121284B2 ## CPC - G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)} - G06T 19/003: {Navigation within 3D models or images} - H04N 13/156: Mixing image signals - H04N 13/282: for generating image signals corresponding to three or more geometrical viewpoints, e.g. multi-view systems - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract There is provided a system and method for integrating a virtual rendering system and a motion control system to provide an augmented three-dimensional reality. There is provided a method for integrating a virtual rendering system and a motion control system for outputting a composite three-dimensional render to a three-dimensional display, the method comprising obtaining, from the motion control system, a robotic three-dimensional camera configuration of a robotic three-dimensional camera in a real environment, programming the virtual rendering system using the robotic three-dimensional camera configuration to correspondingly control a virtual three-dimensional camera in a virtual environment, obtaining a virtually rendered three-dimensional feed using the virtual three-dimensional camera, capturing a video capture three-dimensional feed using the robotic three-dimensional camera, rendering the composite three-dimensional render by processing the feeds, and outputting the composite three-dimensional render to the three-dimensional display. ## Claims ### Claim 1 A method for integrating a virtual rendering system and a motion control system to output a composite three-dimensional render to a three-dimensional display, the method comprising: obtaining, from the motion control system, a first three-dimensional camera configuration, wherein the first three-dimensional camera configuration includes camera position data, camera field of view orientation data, camera movement data and camera lens characteristics data; programming the virtual rendering system using the first three-dimensional camera configuration to correspondingly control a first virtual three-dimensional camera in a virtual environment, wherein the virtual environment includes virtual objects without corresponding real objects in a real environment; obtaining, from the virtual rendering system, a first virtually rendered three-dimensional feed of the virtual environment and the virtual objects using the first virtual three-dimensional camera; capturing, from a video capture system, a first video capture three-dimensional feed of the real environment using the first three-dimensional camera configuration; rendering the composite three-dimensional render by processing the first virtually rendered three-dimensional feed and the first video capture three-dimensional feed; and outputting the composite three-dimensional render to the three-dimensional display; wherein outputting the composite three-dimensional render includes simulating a virtual person in one or more scenarios in the real environment having real elements for strategy analysis, and providing a camera fly-by view for each of the one or more scenarios, wherein the virtual person corresponds to a real person in the real environment. ### Claim 2 (depends on 1) The method of claim 1, wherein the first three-dimensional camera configuration includes a first motion path in the real environment, and the first motion path includes a three-dimensional position with respect to time. ### Claim 3 (depends on 1) The method of claim 1, wherein the first three-dimensional camera configuration includes a first motion path in the real environment, and the first motion path includes a camera orientation or field of view with respect to time. ### Claim 4 (depends on 1) The method of claim 1, wherein the camera lens characteristics data includes three-dimensional camera lens parameters. ### Claim 5 (depends on 1) The method of claim 1, wherein the first three-dimensional camera configuration includes a first motion path in the real environment, and the first motion path tracks a path of a first virtual object of the virtual objects. ### Claim 6 (depends on 1) The method of claim 1, wherein the first three-dimensional camera configuration includes a first motion path in the real environment, and the first motion path is based on a predetermined path. ### Claim 7 (depends on 1) The method of claim 1, wherein the first three-dimensional camera configuration includes a first motion path in the real environment, and the first motion path is based on a previously recorded path. ### Claim 8 (depends on 1) The method of claim 1, wherein the first three-dimensional camera configuration includes a first motion path in the real environment, and the first motion path is based on a video analysis using the video capture system. ### Claim 9 (depends on 1) The method of claim 1, wherein the first three-dimensional camera configuration includes a first motion path in the real environment, and wherein the method further comprising prior to the programming of the virtual rendering system, modifying the first motion path according to a user input. ### Claim 10 A rendering controller for outputting a composite three-dimensional render to a three-dimensional display, the rendering controller comprising: a processor configured to: obtain, from a motion control system, a first three-dimensional camera configuration, wherein the first three-dimensional camera configuration includes camera position data, camera field of view orientation data, camera movement data and camera lens characteristics data; program a virtual rendering system using the first three-dimensional camera configuration to correspondingly control a first virtual three-dimensional camera in a virtual environment, wherein the virtual environment includes virtual objects without corresponding real objects in a real environment; obtain, from the virtual rendering system, a first virtually rendered three-dimensional feed of the virtual environment and the virtual objects using the first virtual three-dimensional camera; capture, from a video capture system, a first video capture three-dimensional feed of the real environment using the first three-dimensional camera configuration; render the composite three-dimensional render by processing the first virtually rendered three-dimensional feed and the first video capture three-dimensional feed; and output the composite three-dimensional render to the three-dimensional display; wherein outputting the composite three-dimensional render includes simulating a virtual person in one or more scenarios in the real environment having real elements for strategy analysis, and providing a camera fly-by view for each of the one or more scenarios, wherein the virtual person corresponds to a real person in the real environment. ### Claim 11 (depends on 10) The rendering controller of claim 10, wherein the first three-dimensional camera configuration includes a first motion path in the real environment, and the first motion path includes a three-dimensional position with respect to time. ### Claim 12 (depends on 10) The rendering controller of claim 10, wherein the first three-dimensional camera configuration includes a first motion path in the real environment, and the first motion path includes a camera orientation or field of view with respect to time. ### Claim 13 (depends on 10) The rendering controller of claim 10, wherein the first three-dimensional camera configuration includes three-dimensional camera lens parameters. ### Claim 14 (depends on 10) The rendering controller of claim 10, wherein the first three-dimensional camera configuration includes a first motion path in the real environment, and the first motion path tracks a path of a first virtual object of virtual objects. ### Claim 15 (depends on 10) The rendering controller of claim 10, wherein the first three-dimensional camera configuration includes a first motion path in the real environment, and the first motion path is based on a predetermined path. ### Claim 16 (depends on 10) The rendering controller of claim 10, wherein the first three-dimensional camera configuration includes a first motion path in the real environment, and the first motion path is based on a previously recorded path. ### Claim 17 (depends on 10) The rendering controller of claim 10, wherein the first three-dimensional camera configuration includes a first motion path in the real environment, and the first motion path is based on a video analysis using the video capture system. ### Claim 18 (depends on 10) The rendering controller of claim 10, wherein the first three-dimensional camera configuration includes a first motion path in the real environment, and wherein prior to the programming of the virtual rendering system, the processor is configured to modify the first motion path according to a user input.