# US 11,897,606 B2 · System and methods for improved aerial mapping with aerial vehicles > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,897,606 B2 - Document ID: US11897606B2 - Status: Granted - Kind: B2 - Application number: 17/382,747 - Filed: 2021-07-22 - Published: 2021-11-11 - Granted: 2024-02-13 - Earliest claimed: 2017-02-02 - Assignees: DroneDeploy, Inc. - Applicants: DroneDeploy, Inc. - Inventors: Jonathan James Millin; Nicholas Pilkington; Devin Lane; Christopher Sullivan; Michael Winn - Web page: https://patentgrep.com/patent/US11897606B2 ## CPC - G06T 17/05: Geographic models - G01C 11/34: Aerial triangulation - G05D 1/0094: {involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target} - G05D 1/689: Pointing payloads towards fixed or moving targets (positioning towed, pushed or suspended implements G05D1/672) - G06V 20/13: Satellite images - G08G 5/21: located onboard the aircraft - G08G 5/26: Transmission of traffic-related information between aircraft and ground stations - G08G 5/32: for flight plan preparation - G08G 5/55: for a single aircraft - G08G 5/57: for unmanned aircraft - G08G 5/59: in accordance with predefined flight zones, e.g. to avoid prohibited zones - G08G 5/74: for monitoring terrain - B64U 2101/32: for cartography or topography - B64U 2201/10: autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS] - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S) - G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93}) - B64U: UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR ## Abstract A method for image generation, preferably including: generating a set of mission parameters for a UAV mission of the UAV associated with aerial scanning of a region of interest; controlling the UAV to perform the mission; generating an image subassembly corresponding to the mission; and/or rendering the image subassembly at a display. ## Claims ### Claim 1 A system comprising: a mobile user device communicatively connected to a camera and configured to receive images of a geographic region from the camera, the mobile device comprising a memory and a processor, wherein the processor is configured to: determine a set of features for each of the images; determine a temporal window of images corresponding to a limited number of the images which are most recently captured or processed, wherein the temporal window of images are captured by the camera and stored at the memory of the mobile user device; and based on the temporal window, determine a refined camera pose associated with the image and a set of feature positions, comprising: for each feature of the set of features, determining a set of matches within the temporal window of images; based on the set of matches, determining a set of estimated positions; and performing a bundle adjustment to refine the set of estimated positions using only images within the temporal window of images to reduce processing requirements of the processor. ### Claim 2 (depends on 1) The system of claim 1, wherein the processor is further configured to determine a terrain homogeneity score based on a size of the set of features, wherein the temporal window of images is determined based on both a processing capacity and the terrain homogeneity score. ### Claim 3 (depends on 1) The system of claim 1, wherein the processor is further configured to: generate an orthomosaic map based on the set of feature positions; and provide the orthomosaic map at a display of the mobile user device. ### Claim 4 (depends on 3) The system of claim 3, further comprising an aircraft, the camera located onboard the aircraft; wherein the orthomosaic map is provided at the display during aircraft flight, wherein the mobile user device is configured to: receive user control inputs based on the orthomosaic map and wirelessly control the aircraft based on the user control inputs. ### Claim 5 (depends on 4) The system of claim 4, wherein the wireless control of the aircraft comprises modifying parameters of an aircraft mission during execution of the mission. ### Claim 6 (depends on 3) The system of claim 3, wherein the processor is further configured to generate a secondary inspection plan for the geographic region based on the orthomosaic map. ### Claim 7 (depends on 6) The system of claim 6, wherein the inspection plan comprises a mission plan for a secondary flight of an aircraft. ### Claim 8 (depends on 3) The system of claim 3, wherein the processor is further configured to: generate an analysis based on the orthomosaic map; and at the display, provide a graphical annotation of the orthomosaic map based on the analysis. ### Claim 9 (depends on 8) The system of claim 8, wherein the analysis comprises a population analysis for a group of organisms within the geographic region. ### Claim 10 (depends on 1) The system of claim 1, wherein performing the bundle adjustment to refine the set of estimated positions comprises minimizing a reprojection error of the set of estimated positions, wherein the reprojection error is not minimized with respect to a subset of camera position parameters. ### Claim 11 A method comprising: at a mobile user device, receiving an image from a camera at a receipt time; determining a set of features based on the image; determining a temporal window of aerial images corresponding to a limited number of the images which are most recently captured or processed within a set of historical images from the camera stored at the mobile user device; for each feature of the set of features, determining a set of matches within the temporal window of aerial images, determining a set of feature positions for each feature of the set of features; at the mobile user device, generating an orthomosaic image based on the set of feature positions; and at the mobile user device, displaying the orthomosaic image in real time relative to the receipt time. ### Claim 12 (depends on 11) The method of claim 11, wherein the camera is wirelessly connected to the mobile user device. ### Claim 13 (depends on 12) The method of claim 12, wherein the camera is arranged onboard an aircraft which is controlled by the mobile user device based on the orthomosaic image. ### Claim 14 (depends on 13) The method of claim 13, further comprising, at the mobile user device, modifying mission parameters of a mission for the aircraft during execution of the mission. ### Claim 15 (depends on 11) The method of claim 11, further comprising: at the mobile user device, generating a mission plan for an imaging aircraft based on the orthomosaic image. ### Claim 16 (depends on 11) The method of claim 11, further comprising: generating an analysis based on the orthomosaic image; and graphically annotating the orthomosaic image based on the analysis. ### Claim 17 (depends on 16) The method of claim 16, wherein the analysis comprises a plant health analysis. ### Claim 18 (depends on 11) The method of claim 11, wherein the orthomosaic image comprises a 2D representation of a scene captured by the set of historical aerial images.