# US 12,002,161 B2 · Methods and apparatus for building a three-dimensional model from multiple data sets > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,002,161 B2 - Document ID: US12002161B2 - Status: Granted - Kind: B2 - Application number: 16/231,090 - Filed: 2018-12-21 - Published: 2019-04-25 - Granted: 2024-06-04 - Earliest claimed: 2012-06-05 - Assignees: Apple Inc. - Applicants: Apple Inc. - Inventors: James A. Howard; Christopher Blumenberg - Web page: https://patentgrep.com/patent/US12002161B2 ## CPC - G06T 19/00: Manipulating three-dimensional [3D] models or images for computer graphics - G01C 21/3638: {including 3D objects and buildings (three dimensional [3D] modelling, e.g. data description of 3D objects G06T17/00; geographic models G06T17/05)} - G06T 15/04: Texture mapping - G06T 15/205: {Image-based rendering} - G06T 17/20: Finite element generation, e.g. wire-frame surface description, {tesselation} - G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S) ## Abstract Methods and apparatus for a map tool displaying a three-dimensional view of a map based on a three-dimensional model of the surrounding environment. The three-dimensional map view of a map may be based on a model constructed from multiple data sets, where the multiple data sets include mapping information for an overlapping area of the map displayed in the map view. For example, one data set may include two-dimensional data including object footprints, where the object footprints may be extruded into a three-dimensional object based on data from a data set composed of three-dimensional data. In this example, the three-dimensional data may include height information that corresponds to the two-dimensional object, where the height may be obtained by correlating the location of the two-dimensional object within the three-dimensional data. ## Claims ### Claim 1 A non-transitory, computer-readable storage medium storing program instructions, wherein the program instructions are computer-executable to implement: providing for display a two-dimensional representation of a map according to two-dimensional mapping information that defines a footprint of a building; in response to input indicating an area of the map: dividing the area into a plurality of segments defined in terms of two-dimensional space; identifying three-dimensional mapping information corresponding to the area of the map, wherein the three-dimensional mapping information comprises mesh data that indicates height information corresponding to multiple locations within the area of the map, wherein the three-dimensional mapping information is received from a map service; correlating individual ones of the plurality of segments with the height information indicated by the mesh data; generating a three-dimensional model of the area based at least in part on extruding the footprint of the building, wherein the extruding comprises (a) extending, using the height information, the plurality of segments in three-dimensional space according to the correlating and (b) performing a smoothing operation to merge at least two adjacent segments whose height difference is within a threshold; and providing for display a three-dimensional view of the area based at least in part on the three-dimensional model. ### Claim 2 (depends on 1) The non-transitory, computer-readable storage medium of claim 1, wherein the plurality of segments comprises a plurality of grid segments of the area of the two-dimensional mapping information, and wherein the plurality of grid segments are of a same size. ### Claim 3 (depends on 1) The non-transitory, computer-readable storage medium of claim 1, wherein the program instructions are computer-executable to further implement: repeating said correlating for a plurality of locations within a segment to determine a plurality of height values corresponding to the plurality of locations; wherein said extending is based on an average value of the plurality of height values. ### Claim 4 (depends on 1) The non-transitory, computer-readable storage medium of claim 1, wherein the height information comprises a plurality of height values within a segment, and wherein said generating the three-dimensional model comprises: selecting a height value from among the plurality of height values of the segment to generate the three-dimensional model. ### Claim 5 (depends on 1) The non-transitory, computer-readable storage medium of claim 1, wherein: the program instructions are computer-executable to further implement: receiving the two-dimensional mapping information that indicates the multiple locations, the two-dimensional mapping information is vector data and not a raster image, and the three-dimensional mapping information is vector data and not a raster image. ### Claim 6 (depends on 1) The non-transitory, computer-readable storage medium of claim 1, wherein the program instructions are computer-executable to further implement: rendering a map view based on the three-dimensional model; wherein the three-dimensional model is a simplified version of the three-dimensional mapping information. ### Claim 7 (depends on 6) The non-transitory, computer-readable storage medium of claim 6, wherein the program instructions are computer-executable to further implement: receiving a navigation instruction corresponding to the map view; and rendering, using the map view and without using updated three-dimensional mapping information, an updated map view. ### Claim 8 A method, comprising: performing, by one or more computing devices: providing for display a two-dimensional representation of a map according to two-dimensional mapping information that defines a footprint of a building; in response to input indicating an area of the map: dividing the area into a plurality of segments defined in terms of two-dimensional space; identifying three-dimensional mapping information corresponding to the area of the map, wherein the three-dimensional mapping information comprises mesh data that indicates height information corresponding to multiple locations within the area of the map, wherein the three-dimensional mapping information is received from a map service; correlating individual ones of the plurality of segments with the height information indicated by the mesh data; generating a three-dimensional model of the area based at least in part on extruding the footprint of the building, wherein the extruding comprises (a) extending, using the height information, the plurality of segments in three-dimensional space according to the correlating and (b) performing a smoothing operation to merge at least two adjacent segments whose height difference is within a threshold; and providing for display a three-dimensional view of the area based at least in part on the three-dimensional model. ### Claim 9 (depends on 8) The method of claim 8, wherein the plurality of segments comprises a plurality of grid segments of the area of the two-dimensional mapping information, and wherein the plurality of grid segments are of a same size. ### Claim 10 (depends on 8) The method of claim 8, further comprising: repeating said correlating for a plurality of locations within a segment to determine a plurality of height values corresponding to the plurality of locations; wherein said extending is based on an average value of the plurality of height values. ### Claim 11 (depends on 8) The method of claim 8, wherein the height information comprises a plurality of height values within a segment, and wherein said generating the three-dimensional model comprises: selecting a height value from among the plurality of height values of the segment to generate the three-dimensional model. ### Claim 12 (depends on 8) The method of claim 8, further comprising: receiving the two-dimensional mapping information that indicates the multiple locations, wherein the two-dimensional mapping information is vector data and not a raster image, and wherein the three-dimensional mapping information is vector data and not a raster image. ### Claim 13 (depends on 8) The method of claim 8, further comprising: rendering a map view based on the three-dimensional model; wherein the three-dimensional model is a simplified version of the three-dimensional mapping information. ### Claim 14 (depends on 13) The method of claim 13, further comprising: receiving a navigation instruction corresponding to the map view; and rendering, using the map view and without using updated three-dimensional mapping information, an updated map view. ### Claim 15 A system, comprising: at least one processor; and a memory comprising program instructions, wherein the program instructions are executable by the at least one processor to: provide for display a two-dimensional representation of a map according to two-dimensional mapping information that defines a footprint of a building; in response to input indicating an area of the map: divide the area into a plurality of segments defined in terms of two-dimensional space; identify three-dimensional mapping information corresponding to the area of the map, wherein the three-dimensional mapping information comprises mesh data that indicates height information corresponding to multiple locations within the area of the map, wherein the three-dimensional mapping information is received from a map service; correlate individual ones of the plurality of segments with the height information indicated by the mesh data; generate a three-dimensional model of the area based at least in part on extruding the footprint of the building, wherein the extruding comprises extending, using the height information, the plurality of segments in three-dimensional space according to the correlation, wherein an object indicated in the two-dimensional mapping information is generated in a lower level of granularity in the three-dimensional model; and provide for display a three-dimensional view of the area based at least in part on the three-dimensional model. ### Claim 16 (depends on 15) The system of claim 15, wherein the plurality of segments comprises a plurality of grid segments of the area of the two-dimensional mapping information, and wherein the plurality of grid segments are of a same size. ### Claim 17 (depends on 15) The system of claim 15, wherein the program instructions are further executable by the at least one processor to: repeat said correlation for a plurality of locations within a segment to determine a plurality of height values corresponding to the plurality of locations; wherein said extending is based on an average value of the plurality of height values. ### Claim 18 (depends on 15) The system of claim 15, wherein the height information comprises a plurality of height values within a segment, and wherein, to generate the three-dimensional model, the program instructions are executable by the at least one processor to: select a height value from among the plurality of height values of the segment to generate the three-dimensional model. ### Claim 19 (depends on 15) The system of claim 15, wherein: the program instructions are further executable by the at least one processor to: receive the two-dimensional mapping information that indicates the multiple locations, the two-dimensional mapping information is vector data and not a raster image, and the three-dimensional mapping information is vector data and not a raster image. ### Claim 20 (depends on 15) The system of claim 15, wherein to provide the three-dimensional view, the program instructions are further executable by the at least one processor to: render a map view based on the three-dimensional model and one or more map textures received from the map service.