# US 11,440,657 B2 · Aerial vehicles with machine vision > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,440,657 B2 - Document ID: US11440657B2 - Status: Granted - Kind: B2 - Application number: 16/240,913 - Filed: 2019-01-07 - Published: 2019-08-01 - Granted: 2022-09-13 - Earliest claimed: 2018-01-29 - Assignees: GE Aviation Systems LLC - Applicants: Ge Aviation Systems Limited - Inventors: Stefan Alexander Schwindt - Web page: https://patentgrep.com/patent/US11440657B2 ## CPC - G08G 5/21: located onboard the aircraft - B64D 45/04: Landing aids; Safety measures to prevent collision with earth's surface - G06V 20/13: Satellite images - G06V 20/17: taken from planes or by drones - G08G 5/51: for control when on the ground, e.g. taxiing or rolling - G08G 5/54: for approach or landing - G08G 5/55: for a single aircraft - G08G 5/57: for unmanned aircraft - G08G 5/74: for monitoring terrain - B64D 47/08: Arrangements of cameras - B64U 2201/10: autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS] - G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93}) - B64D: EQUIPMENT FOR FITTING IN OR TO AIRCRAFT; FLIGHT SUITS; PARACHUTES; ARRANGEMENT OR MOUNTING OF POWER PLANTS OR PROPULSION TRANSMISSIONS IN AIRCRAFT - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - B64U: UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR ## Abstract An aerial vehicle is provided. The aerial vehicle can include a plurality of sensors mounted thereon, an avionics system configured to operate at least a portion of the aerial vehicle, and a machine vision controller in operative communication with the avionics system and the plurality of sensors. The machine vision controller is configured to perform a method. The method includes obtaining sensor data from at least one sensor of the plurality of sensors, determining performance data from the avionic system or an additional sensor of the plurality of sensors, processing the sensor data based on the performance data to compensate for movement of the unmanned aerial vehicle, identifying at least one geographic indicator based on processing the sensor data, and determining a geographic location of the aerial vehicle based on the at least one geographic indicator. ## Claims ### Claim 1 An aerial vehicle, comprising: a plurality of sensors mounted thereon; an avionics system configured to operate at least a portion of the aerial vehicle; and a machine vision controller in operative communication with the avionics system and the plurality of sensors, the machine vision controller configured to perform a method, the method comprising: obtaining sensor data from at least one sensor of the plurality of sensors, wherein the sensor data is associated with a proximal location of the aerial vehicle and the proximal location is within sensor range of the aerial vehicle; determining performance data associated with the aerial vehicle; processing the sensor data based on the performance data to compensate for movement of the unmanned aerial vehicle; identifying at least one geographic indicator based on processing the sensor data, wherein identifying the at least one geographical indicator comprises comparing the sensor data to a predetermined set of sensor data, identifying an obstacle associated with the sensor data based on the comparing, and triggering an alert indicating identification of the obstacle; determining whether the obstacle is moving; and determining a geographic location of the aerial vehicle based on the at least one geographic indicator, wherein when the obstacle is determined to be moving, delaying the triggering of the alert. ### Claim 2 (depends on 1) The aerial vehicle of claim 1, wherein processing the sensor data comprises: determining a height of a plurality of geographic features in the proximal location; comparing the plurality of geographic features with a database identifying a plurality of predetermined geographic indicators associated with known landing locations, the database including a height identifier for each of the plurality of predetermined geographic indicators; wherein determining the geographic location of the aerial vehicle comprises identifying the geographic location of the aerial vehicle based at least in part on the height identifier for the at least one geographic indicator and the height of a corresponding geographic feature. ### Claim 3 (depends on 1) The aerial vehicle of claim 1, wherein the sensor data is based on still images or video recording by an image capture device located on the aerial vehicle. ### Claim 4 (depends on 1) The aerial vehicle of claim 1, wherein the sensor data is based on remote sensing associated with a laser, Light Detection and Ranging (LIDAR), or radar. ### Claim 5 (depends on 1) The aerial vehicle of claim 1, wherein the processing of the sensor data comprises processing the sensor data to reduce distortion based on the performance data associated with the aerial vehicle. ### Claim 6 (depends on 5) The aerial vehicle of claim 5, wherein processing the sensor data to reduce distortion comprises: determining a crosswind value; estimating an approach vector based on the crosswind value and the performance data; and translating the sensor data based on the approach vector. ### Claim 7 (depends on 1) The aerial vehicle of claim 1, wherein identifying the at least one geographical indicator comprises: comparing the sensor data to a model of the proximal area; and matching a first geographic indicator associated with the sensor data to at least one second geographical indicator associated with the model of the proximal area. ### Claim 8 (depends on 1) The aerial vehicle of claim 1, wherein identifying the at least one geographical indicator comprises: comparing the sensor data to a predetermined set of sensor data; and identifying an obstacle associated with the sensor data based on the comparing. ### Claim 9 (depends on 1) The aerial vehicle of claim 1, wherein the method further comprises: determining that the geographic location of the aerial vehicle is absent from a flight plan of the aerial vehicle; and instructing the avionics system to perform a corrective maneuver in response to the determining that the geographic location of the aerial vehicle is absent. ### Claim 10 (depends on 1) The aerial vehicle of claim 1, wherein the method further comprises: determining, based on the performance data, whether the aerial vehicle is attempting a landing; determining whether the at least one geographic indicator is associated with an appropriate landing area for the aerial vehicle; and in response to determining that the aerial vehicle is attempting a landing and that the at least one geographic indicator is associated with an appropriate landing area, providing a visual indication that the geographic location corresponds to an appropriate landing area. ### Claim 11 (depends on 10) The aerial vehicle of claim 10, wherein the method further comprises: in response to determining that the aerial vehicle is not attempting a landing and that the at least one geographic indicator is associated with an appropriate landing area, determining that the visual indication should not be displayed. ### Claim 12 (depends on 1) The aerial vehicle of claim 1, wherein identifying the at least one geographical indicator comprises: comparing the sensor data to a predetermined set of sensor data; identifying an obstacle associated with the sensor data based on the comparing; and triggering an alert indicating identification of the obstacle. ### Claim 13 (depends on 12) The aerial vehicle of claim 12, further comprising: when the obstacle is determined to be moving, the alert triggered is one of a soft alert and a high alert depending on whether the obstacle is deemed hazardous, not moving as expected, or not following an expected pattern. ### Claim 14 (depends on 1) The aerial vehicle of claim 1, wherein the at least one geographic indicator is a building surrounding a landing area. ### Claim 15 (depends on 1) The aerial vehicle of claim 1, wherein the at least one geographic indicator is as natural feature. ### Claim 16 (depends on 1) The aerial vehicle of claim 1, wherein the performance data further includes at least one of a rate of ascent and a rate of descent. ### Claim 17 A method of locating an aerial vehicle relative to a proximal location, the method comprising: obtaining sensor data from one or more optical sensors mounted on the aerial vehicle, wherein the sensor data is associated with the proximal location and the proximal location is within sensor range of the aerial vehicle; determining performance data from one or more additional sensors mounted on the aerial vehicle; determining, based on the performance data, whether the aerial vehicle is attempting a landing; processing the sensor data based on the performance data to compensate for movement of the unmanned aerial vehicle; identifying at least one geographic indicator based on processing the sensor data; determining whether the at least one geographic indicator is associated with an appropriate landing area for the aerial vehicle; determining a geographic location of the aerial vehicle based on the at least one geographic indicator; and in response to determining that the aerial vehicle is attempting a landing and that the at least one geographic indicator is associated with an appropriate landing area, providing a visual indication that the geographic location corresponds to an appropriate landing area. ### Claim 18 (depends on 17) The method of claim 17, wherein the method further comprises: in response to determining that the aerial vehicle is not attempting a landing and that the at least one geographic indicator is associated with an appropriate landing area, determining that the visual indication should not be displayed. ### Claim 19 An aerial vehicle, comprising: a plurality of sensors mounted thereon; an avionics system configured to operate at least a portion of the aerial vehicle; and a machine vision controller in operative communication with the avionics system and the plurality of sensors, the machine vision controller configured to perform a method, the method comprising: obtaining sensor data from at least one sensor of the plurality of sensors, wherein the sensor data is associated with a proximal location of the aerial vehicle and the proximal location is within sensor range of the aerial vehicle; determining performance data associated with the aerial vehicle; processing the sensor data based on the performance data to compensate for movement of the unmanned aerial vehicle, wherein the processing comprises: determining a crosswind value; estimating an approach vector based on the crosswind value and the performance data; and translating the sensor data based on the approach vector; identifying at least one geographic indicator based on processing the sensor data; and determining a geographic location of the aerial vehicle based on the at least one geographic indicator.