# US 12,217,456 B2 · Guided batching > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,217,456 B2 - Document ID: US12217456B2 - Status: Granted - Kind: B2 - Application number: 18/590,815 - Filed: 2024-02-28 - Published: 2024-09-26 - Granted: 2025-02-04 - Earliest claimed: 2019-11-25 - Assignees: Lyft, Inc. - Applicants: Lyft, Inc. - Inventors: Long Chen; Benjamin Alex Haines; Luca Del Pero - Web page: https://patentgrep.com/patent/US12217456B2 ## CPC - G06T 7/74: {involving reference images or patches} - G01C 21/3602: {Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera} - G01C 21/3638: {including 3D objects and buildings (three dimensional [3D] modelling, e.g. data description of 3D objects G06T17/00; geographic models G06T17/05)} - G01C 21/3804: {Creation or updating of map data} - G06F 16/583: using metadata automatically derived from the content - G06F 18/2323: based on graph theory, e.g. minimum spanning trees [MST] or graph cuts - G06F 18/251: {of input or preprocessed data} - G06T 7/136: involving thresholding - G06T 7/579: from motion - G06T 7/97: {Determining parameters from multiple pictures (depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85)} - G06T 17/05: Geographic models - G06V 10/26: Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion - G06V 10/7635: {based on graphs, e.g. graph cuts or spectral clustering} - G06V 10/803: {of input or preprocessed data} - G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle - H04N 23/698: for achieving an enlarged field of view, e.g. panoramic image capture - G06T 2207/10016: Video; Image sequence - G06T 2207/10028: Range image; Depth image; 3D point clouds - G06T 2207/20072: Graph-based image processing - G06T 2207/30236: Traffic on road, railway or crossing - G06T 2207/30244: Camera pose - G06T 2207/30256: Lane; Road marking - G06V 10/16: {using multiple overlapping images; Image stitching} - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract The present invention provides a method of generating a robust global map using a plurality of limited field-of-view cameras to capture an environment. Provided is a method for generating a three-dimensional map comprising: receiving a plurality of sequential image data wherein each of the plurality of sequential image data comprises a plurality of sequential images, further wherein the plurality of sequential images is obtained by a plurality of limited field-of-view image sensors; determining a pose of each of the plurality of sequential images of each of the plurality of sequential image data; determining one or more overlapping poses using the determined poses of the sequential image data; selecting at least one set of images from the plurality of sequential images wherein each set of images are determined to have overlapping poses; and constructing one or more map portions derived from each of the at least one set of images. ## Claims ### Claim 1 A computer-implemented method comprising: determining, by a computing system, a first subset of a set of image data based on a location of a point of interest; determining, by the computing system, one or more second subsets of the set of image data based on a radius around the location of the point of interest; generating, by the computing system, a group of image data comprising the first subset and the one or more second subsets; and generating, by the computing system, a map portion associated with the point of interest based on the group of image data. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, wherein the first subset is a central subset of the set of image data, and wherein the first subset corresponds with a center location of a geographic area comprising the point of interest. ### Claim 3 (depends on 1) The computer-implemented method of claim 1, further comprising: filtering, by the computing system, the group of image data based on visual similarity with the first subset. ### Claim 4 (depends on 3) The computer-implemented method of claim 3, wherein the filtering comprises: determining, by the computing system, a first sub-subset of the first subset and a second sub-subset of the one or more second subsets; and comparing, by the computing system, the first sub-subset of the first subset and the second sub-subset of the one or more second subsets. ### Claim 5 (depends on 3) The computer-implemented method of claim 3, wherein the filtering is based on a set of operations run in order of computational complexity. ### Claim 6 (depends on 1) The computer-implemented method of claim 1, wherein the first subset includes a first image frame that overlaps with a second image frame of the one or more second subsets. ### Claim 7 (depends on 1) The computer-implemented method of claim 1, further comprising: connecting, by the computing system, the map portion to a global map based on a determination of a correlation point for the map portion and the global map. ### Claim 8 (depends on 1) The computer-implemented method of claim 1, further comprising: causing, by the computing system, an adjustment to the map portion based on a bundle adjustment of a global map comprising the map portion. ### Claim 9 (depends on 1) The computer-implemented method of claim 1, wherein generating the map portion comprises: generating, by the computing system, a photographic rendering of a scene at the point of interest based on the first subset, the one or more second subsets, and metadata associated with the first subset and the one or more second subsets. ### Claim 10 (depends on 1) The computer-implemented method of claim 1, wherein the first subset depicts a first environmental condition and the one or more second subset depict a second environment condition. ### Claim 11 A system comprising: at least one processor; and a memory storing instructions that, when executed, cause the system to perform operations comprising: determining a first subset of a set of image data based on a location of a point of interest; determining one or more second subsets of the set of image data based on a radius around the location of the point of interest; generating a group of image data comprising the first subset and the one or more second subsets; and generating a map portion associated with the point of interest based on the group of image data. ### Claim 12 (depends on 11) The system of claim 11, wherein the first subset is a central subset of the set of image data, and wherein the first subset corresponds with a center location of a geographic area comprising the point of interest. ### Claim 13 (depends on 11) The system of claim 11, the operations further comprising: filtering the group of image data based on visual similarity with the first subset. ### Claim 14 (depends on 13) The system of claim 13, wherein the filtering comprises: determining a first sub-subset of the first subset and a second sub-subset of the one or more second subsets; and comparing the first sub-subset of the first subset and the second sub-subset of the one or more second subsets. ### Claim 15 (depends on 13) The system of claim 13, wherein the filtering is based on a set of operations run in order of computational complexity. ### Claim 16 A non-transitory computer-readable storage medium including instructions that, when executed, cause a computing system to perform operations comprising: determining a first subset of a set of image data based on a location of a point of interest; determining one or more second subsets of the set of image data based on a radius around the location of the point of interest; generating a group of image data comprising the first subset and the one or more second subsets; and generating a map portion associated with the point of interest based on the group of image data. ### Claim 17 (depends on 16) The non-transitory computer-readable storage medium of claim 16, wherein the first subset is a central subset of the set of image data, and wherein the first subset corresponds with a center location of a geographic area comprising the point of interest. ### Claim 18 (depends on 16) The non-transitory computer-readable storage medium of claim 16, the operations further comprising: filtering the group of image data based on visual similarity with the first subset. ### Claim 19 (depends on 18) The non-transitory computer-readable storage medium of claim 18, wherein the filtering comprises: determining a first sub-subset of the first subset and a second sub-subset of the one or more second subsets; and comparing the first sub-subset of the first subset and the second sub-subset of the one or more second subsets. ### Claim 20 (depends on 18) The non-transitory computer-readable storage medium of claim 18, wherein the filtering is based on a set of operations run in order of computational complexity.