# US 12,182,930 B2 · Enclosed multi-view visual media representation > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,182,930 B2 - Document ID: US12182930B2 - Status: Granted - Kind: B2 - Application number: 18/173,741 - Filed: 2023-02-23 - Published: 2023-06-22 - Granted: 2024-12-31 - Earliest claimed: 2021-06-08 - Assignees: Fyusion, Inc. - Applicants: Fyusion, Inc. - Inventors: Nico Gregor Sebastian Blodow; Martin Saelzle; Matteo Munaro; Krunal Ketan Chande; Rodrigo Ortiz Cayon; Stefan Johannes Josef Holzer - Web page: https://patentgrep.com/patent/US12182930B2 ## CPC - G06T 15/20: Perspective computation - G06T 17/00: Three-dimensional [3D] modelling for computer graphics - G06T 19/003: {Navigation within 3D models or images} - G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)} - H04N 13/111: Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation - H04N 13/243: using three or more two-dimensional [2D] image sensors - H04N 13/271: wherein the generated image signals comprise depth maps or disparity maps - H04N 13/282: for generating image signals corresponding to three or more geometrical viewpoints, e.g. multi-view systems - G06T 2210/56: Particle system, point based geometry or rendering - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract Images may be captured at an image capture device mounted on an image capture device gimbal capable of rotating the image capture device around a nodal point in one or more dimensions. Each of the plurality of images may be captured from a respective rotational position. The images may be captured by a designated camera that is not located at the nodal point in one or more of the respective rotational positions. A designated three-dimensional point cloud may be determined based on the plurality of images. The designated three-dimensional point cloud may include a plurality of points each having a respective position in a virtual three-dimensional space. ## Claims ### Claim 1 A method comprising: identifying a plurality of images captured at an image capture device mounted on an image capture device gimbal, the image capture device gimbal capable of rotating the image capture device around a nodal point in three dimensions, each of the plurality of images being captured from a respective rotational position, the plurality of images being captured by a designated camera that is not located at the nodal point in one or more of the respective rotational positions; generating one or more synthetic images using a designated three-dimensional model having a plurality of points in a virtual three-dimensional space, each of the synthetic images being generated from a respective virtual viewpoint within the virtual three-dimensional space, the synthetic images generated using positions of the plurality of images captured by the designated camera to correct for parallax errors caused by the images being captured at different relative positions from the nodal point in various gimbal rotational positions; and storing the one or more synthetic images on a storage device. ### Claim 2 (depends on 1) The method of claim 1, further comprising: determining a plurality of respective positions for the plurality of images, a designated one of the respective positions identifying a position relative to the nodal point for a designated one of the plurality of images; determining the designated three-dimensional model using the plurality of images and the plurality of respective positions, the designated three-dimensional model including a plurality of points each having respective positions in the virtual three-dimensional space. ### Claim 3 (depends on 2) The method of claim 2, wherein determining the designated three-dimensional model comprises determining a plurality of three-dimensional models corresponding with respective ones of the plurality of images. ### Claim 4 (depends on 3) The method of claim 3, wherein the designated camera is located at a designated position relative to the nodal point, the designated position having a translation component. ### Claim 5 (depends on 4) The method of claim 4, wherein the designated three-dimensional model is determined at least in part by numerically estimating a value for the translational component using the plurality of three-dimensional models and the respective rotational positions. ### Claim 6 (depends on 5) The method of claim 5, wherein the designated position has a rotational component, and wherein the designated three-dimensional model is determined at least in part by numerically estimating a value for the rotational component using the plurality of three-dimensional models and the respective rotational positions. ### Claim 7 (depends on 2) The method of claim 2, wherein each of the respective virtual viewpoints is located at the nodal point. ### Claim 8 (depends on 2) The method of claim 2, wherein one or more of the plurality of images is associated with respective depth sensor data captured from a depth sensor located at the image capture device, and wherein the designated three-dimensional model is determined at least in part using the respective depth sensor data. ### Claim 9 (depends on 2) The method of claim 2, wherein the image capture device is configured to capture inertial data via an inertial measurement unit located at the image capture device, the method further comprising identifying one or more of the respective rotational positions based at least in part on the inertial data. ### Claim 10 (depends on 2) The method of claim 2, the method further comprising: generating a multi-view interactive digital media representation using the synthetic images, the multi-view interactive digital media representation being navigable in one or more dimensions. ### Claim 11 (depends on 10) The method of claim 10, wherein navigating the multi-view interactive digital media representation comprises: presenting a first one of the synthetic images on a display screen, receiving user input indicating an adjustment to a rotational perspective, and presenting a second one of the synthetic images on the display screen, the second synthetic image being selected to reflect the adjustment to the rotational perspective. ### Claim 12 (depends on 2) The method of claim 2, wherein identifying the plurality of images comprises transmitting one or more messages between the image capture device and the image capture device gimbal. ### Claim 13 (depends on 12) The method of claim 12, wherein a first one of the messages is transmitted from the image capture device to the image capture device gimbal, the first message indicating a first one of the respective rotational positions. ### Claim 14 (depends on 13) The method of claim 13, wherein a second one of the messages is transmitted from the image capture device gimbal to the image capture device, the second message including position data generated by the image capture device gimbal. ### Claim 15 An image capture device comprising: a designated camera configured to capture a plurality of images while the image capture device is mounted on an image capture device gimbal capable of rotating the image capture device around a nodal point in three dimensions, each of the plurality of images being captured from a respective rotational position, the designated camera not being located at the nodal point in one or more of the respective rotational positions; a processor configured to generating one or more synthetic images using a designated three-dimensional model having a plurality of points in a virtual three-dimensional space, each of the synthetic images being generated from a respective virtual viewpoint within the virtual three-dimensional space, the synthetic images generated using positions of the plurality of images captured by the designated camera to correct for parallax errors caused by the images being captured at different relative positions from the nodal point in various gimbal rotational positions; and a storage device configured to store the one or more synthetic images. ### Claim 16 (depends on 15) The image capture device of claim 15, wherein the processor is further configured to determine a plurality of respective positions for the plurality of images, a designated one of the respective positions identifying a position relative to the nodal point for a designated one of the plurality of images. ### Claim 17 (depends on 16) The image capture device of claim 16, wherein the processor is further configured to determine the designated three-dimensional model using the plurality of images and the plurality of respective positions, the designated three-dimensional model including a plurality of points each having respective positions in the virtual three-dimensional space. ### Claim 18 (depends on 17) The image capture device of claim 17, wherein determining the designated three-dimensional model comprises determining a plurality of three-dimensional models corresponding with respective ones of the plurality of images, and wherein the designated camera is located at a designated position relative to the nodal point, the designated position having a translation component, and wherein the designated three-dimensional model is determined at least in part by numerically estimating a value for the translational component using the plurality of three-dimensional models and the respective rotational positions. ### Claim 19 A non-transitory computer readable medium comprising: computer code for identifying a plurality of images captured at an image capture device mounted on an image capture device gimbal, the image capture device gimbal capable of rotating the image capture device around a nodal point in three dimensions, each of the plurality of images being captured from a respective rotational position, the plurality of images being captured by a designated camera that is not located at the nodal point in one or more of the respective rotational positions; computer code for generating one or more synthetic images using a designated three-dimensional model having a plurality of points in a virtual three-dimensional space, each of the synthetic images being generated from a respective virtual viewpoint within the virtual three-dimensional space, the synthetic images generated using positions of the plurality of images captured by the designated camera to correct for parallax errors caused by the images being captured at different relative positions from the nodal point in various gimbal rotational positions; and computer code for storing the one or more synthetic images on a storage device. ### Claim 20 (depends on 19) The non-transitory computer readable medium of claim 19, comprising: computer code for determining a plurality of respective positions for the plurality of images, a designated one of the respective positions identifying a position relative to the nodal point for a designated one of the plurality of images; computer code for determining the designated three-dimensional model using the plurality of images and the plurality of respective positions, the designated three-dimensional model including a plurality of points each having respective positions in the virtual three-dimensional space.