# US 11,113,882 B2 · Generating immersive trip photograph visualizations > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,113,882 B2 - Document ID: US11113882B2 - Status: Granted - Kind: B2 - Application number: 17/062,079 - Filed: 2020-10-02 - Published: 2021-01-21 - Granted: 2021-09-07 - Earliest claimed: 2018-09-27 - Assignees: Adobe Inc.; Brno University of Technology - Applicants: Adobe Inc. - Inventors: Michal Luká{hacek over (c)}; Zhili Chen; Jan Brejcha; Martin Cadik - Web page: https://patentgrep.com/patent/US11113882B2 ## CPC - G06T 19/003: {Navigation within 3D models or images} - G06F 3/048: Interaction techniques based on graphical user interfaces [GUI] - G06F 16/29: Geographical information databases - G06F 16/583: using metadata automatically derived from the content - G06T 7/579: from motion - G06T 7/73: using feature-based methods - G06V 20/10: Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59) - G06V 20/38: {Outdoor scenes} - G06V 20/70: Labelling scene content, e.g. deriving syntactic or semantic representations - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A user selects a set of photographs from a trip through an environment that he or she desires to present to other people. A collection of photographs, including the set of photographs captured during the trip optionally augmented with additional photographs obtained from another collection, are combined with a terrain model (e.g., a digital elevation model) to extract information regarding the geographic location of each of the photographs within the environment. The collection of photographs are analyzed, considering their geolocation information as well as the photograph content to register the photographs relative to one another. This information for the photographs is compared to the terrain model in order to accurately position the viewpoint for each photograph within the environment. A presentation of the selected photographs within the environment is generated that displays both the selected photographs and synthetic data filled in beyond the edges of the selected photographs. ## Claims ### Claim 1 In a digital medium generation environment, a method implemented by at least one computing device, the method comprising: obtaining a set of photographs captured on a geographical route; receiving a three dimensional terrain model of a geographical area that includes the geographical route; receiving an indication of the geographical area where the set of photographs were captured; determining both a location and an orientation of ones of the set of photographs within the three dimensional terrain model of the geographical area; and generating a sequence of images through the three dimensional terrain model along the geographical route through the geographical area, the sequence of images including multiple photographs in the set of photographs and, between ones of the multiple photographs, images generated from the three dimensional terrain model. ### Claim 2 (depends on 1) The method as described in claim 1, further comprising displaying the sequence of images including pausing for a threshold amount of time to display each of the multiple photographs. ### Claim 3 (depends on 1) The method as described in claim 1, further comprising displaying the sequence of images allowing, for each location where one of the multiple photographs was captured, a viewer to look around the location in a 360 degree view. ### Claim 4 (depends on 1) The method as described in claim 1, further comprising displaying the sequence of images including displaying at least one photograph in the multiple photographs along with data from the three dimensional terrain model that fills in an area surrounding the at least one photograph. ### Claim 5 (depends on 1) The method as described in claim 1, wherein the geographical route includes a first portion that is an actual path taken through the geographical area and a second portion that is an estimate of a path taken through the geographical area. ### Claim 6 (depends on 1) The method as described in claim 1, wherein generating the sequence of images comprises determining the geographical route based on locations where the set of photographs were captured. ### Claim 7 (depends on 1) The method as described in claim 1, wherein the determining comprises: extracting visual features from each photograph in the set of photographs; determining, based on the visual features, both three dimensional location information and orientation information for each photograph in the set of photographs; reconstructing terrain represented in the set of images by generating a point cloud from the set of images; and aligning the reconstructed terrain with the terrain model to fine-tune the dimensional location information and the orientation information for each photograph in the set of photographs. ### Claim 8 In a digital medium generation environment, a method implemented by at least one computing device, the method comprising: obtaining a set of photographs captured on a geographical route; receiving a three dimensional terrain model of a geographical area that includes the geographical route; receiving an indication of the geographical area where the set of photographs were captured; determining both a location and an orientation of ones of the set of photographs within the three dimensional terrain model of the geographical area; and displaying at least one photograph in the set of photographs along with data from the three dimensional terrain model that fills in an area surrounding the at least one photograph. ### Claim 9 (depends on 8) The method as described in claim 8, further comprising generating a sequence of images through the three dimensional terrain model along the geographical route, the sequence of images including multiple photographs in the set of photographs, the displaying including displaying the sequence of images including the at least one photograph as part of the sequence of images. ### Claim 10 (depends on 9) The method as described in claim 9, the displaying including pausing for a threshold amount of time to display each photograph from the set of photographs in the sequence of images. ### Claim 11 (depends on 9) The method as described in claim 9, the sequence of images including, between ones of the multiple photographs, images generated from the three dimensional terrain model. ### Claim 12 (depends on 9) The method as described in claim 9, wherein generating the sequence of images comprises determining the geographical route based on locations where the set of photographs were captured. ### Claim 13 (depends on 8) The method as described in claim 8, wherein the geographical route includes a first portion that is an actual path taken through the geographical area and a second portion that is an estimate of a path taken through the geographical area. ### Claim 14 (depends on 8) The method as described in claim 8, wherein the determining comprises: extracting visual features from each photograph in the set of photographs; determining, based on the visual features, both three dimensional location information and orientation information for each photograph in the set of photographs; reconstructing terrain represented in the set of photographs by generating a point cloud from the set of photographs; and aligning the reconstructed terrain with the terrain model to fine-tune the dimensional location information and the orientation information for each photograph in the set of photographs. ### Claim 15 In a digital medium generation environment, a device comprising: one or more processors; one or more computer-readable storage media having stored thereon multiple instructions that, in response to execution by the one or more processors, cause the one or more processors to: obtain a set of photographs captured on a geographical route; receive a three dimensional terrain model of a geographical area that includes the geographical route; receive an indication of the geographical area where the set of photographs were captured; determine both a location and an orientation of ones of the set of photographs within the three dimensional terrain model of the geographical area; and generate a sequence of images through the three dimensional terrain model along the geographical route through the geographical area, the sequence of images including multiple photographs in the set of photographs and, between ones of the multiple photographs, images generated from the three dimensional terrain model. ### Claim 16 (depends on 15) The device as described in claim 15, wherein the multiple instructions further cause the one or more processors to display the sequence of images including pausing for a threshold amount of time to display each of the multiple photographs. ### Claim 17 (depends on 15) The device as described in claim 15, wherein the multiple instructions further cause the one or more processors to display the sequence of images and allow, for each location where one of the multiple photographs was captured, a viewer to look around the location in a 360 degree view. ### Claim 18 (depends on 15) The device as described in claim 15, wherein the multiple instructions further cause the one or more processors to display the sequence of images including displaying at least one photograph in the multiple photographs along with data from the three dimensional terrain model that fills in an area surrounding the at least one photograph. ### Claim 19 (depends on 15) The device as described in claim 15, wherein the geographical route includes a first portion that is an actual path taken through the geographical area and a second portion that is an estimate of a path taken through the geographical area. ### Claim 20 (depends on 15) The device as described in claim 15, wherein to generate the sequence of images includes to determine the geographical route based on locations where the set of photographs were captured.