# US 12,217,500 B2 · Automatic terrain evaluation of landing surfaces, and associated systems and methods > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,217,500 B2 - Document ID: US12217500B2 - Status: Granted - Kind: B2 - Application number: 18/215,836 - Filed: 2023-06-29 - Published: 2023-10-26 - Granted: 2025-02-04 - Earliest claimed: 2016-08-06 - Assignees: Sz Dji Technology Co., Ltd. - Applicants: Sz Dji Technology Co., Ltd. - Inventors: Zhenyu Zhu; Honghui Zhang; You Zhou - Web page: https://patentgrep.com/patent/US12217500B2 ## CPC - G06V 20/13: Satellite images - B64D 45/08: optical - B64U 60/50: with landing legs - G05D 1/042: {specially adapted for aircraft} - G05D 1/243: Means capturing signals occurring naturally from the environment, e.g. ambient optical, acoustic, gravitational or magnetic signals (using passive navigation aids external to the vehicle G05D1/244; using signals from positioning sensors located off-board the vehicle G05D1/249) - G05D 1/46: Control of position or course in three dimensions [3D] - G05D 1/644: Optimisation of travel parameters, e.g. of energy consumption, journey time or distance - G05D 1/654: Landing (docking at a base station G05D1/661) - G06T 7/593: from stereo images - G06V 20/17: taken from planes or by drones - G06V 20/64: Three-dimensional [3D] objects - B64U 70/90: Launching from or landing on platforms - B64U 2101/32: for cartography or topography - B64U 2201/20: Remote controls - G05D 1/0038: {by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation (images analyzed by a computer and used for automatic navigation G05D1/0246)} - G05D 1/2247: {providing the operator with simple or augmented images from one or more cameras} - G05D 2109/254: {Flying platforms, e.g. multicopters} - G05D 2111/10: Optical signals - G06T 2207/10012: Stereo images - G06T 2207/10028: Range image; Depth image; 3D point clouds - G06T 2207/10032: Satellite or aerial image; Remote sensing - G06T 2207/30181: Earth observation - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - B64D: EQUIPMENT FOR FITTING IN OR TO AIRCRAFT; FLIGHT SUITS; PARACHUTES; ARRANGEMENT OR MOUNTING OF POWER PLANTS OR PROPULSION TRANSMISSIONS IN AIRCRAFT - B64U: UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR - G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract Automatic terrain evaluation of landing surfaces, and associated systems and methods are disclosed herein. A representative method includes receiving a request to land a movable object and, in response to the request, identifying a target landing area on a landing surface based on at least one image of the landing surface obtained by the movable object. The method can further include directing the movable object to land at the target landing area. ## Claims ### Claim 1 A method for controlling landing of a movable object, comprising: receiving a request to land the movable object on a surface; determining flatness or tilt of the surface based on data measured by one or more sensors; and controlling the movable object to land based on the flatness or the tilt, including: in response to determining that the surface is not flat or is tilted, controlling the movable object to move to a location corresponding to another surface or controlling the movable object to hover. ### Claim 2 (depends on 1) The method of claim 1, wherein controlling the movable object to land based on the flatness or the tilt further comprises: determining whether the surface meets one or more landing conditions based on the flatness or the tilt; and controlling the movable object to stop landing when the one or more landing conditions are not met. ### Claim 3 (depends on 2) The method of claim 2, wherein controlling the movable object to stop landing comprises: guiding the movable object to another location to identify a possible landing site. ### Claim 4 (depends on 2) The method of claim 2, wherein controlling the movable object to stop landing comprises: navigating around a landing location. ### Claim 5 (depends on 1) The method of claim 1, wherein controlling the movable object to land based on the flatness or the tilt further comprises: controlling the movable object to continue landing when one or more landing conditions are met. ### Claim 6 (depends on 1) The method of claim 1, further comprising: controlling the movable object to descend to a predetermined height before obtaining the data measured by the one or more sensors. ### Claim 7 (depends on 6) The method of claim 6, wherein the data measured by the one or more sensors is obtained when the movable object is at the predetermined height. ### Claim 8 (depends on 1) The method of claim 1, wherein determining the flatness or the tilt of the surface based on the data measured by the one or more sensors comprises: obtaining a depth map of the surface based on the data measured by the one or more sensors, each pixel of the depth map corresponding to a portion of the surface; and determining the flatness or the tilt of the depth map. ### Claim 9 (depends on 8) The method of claim 8, wherein obtaining the depth map of the surface comprises: performing a stereo matching on images captured by two cameras at the same time or two sequential images captured by a single camera. ### Claim 10 (depends on 8) The method of claim 8, wherein the depth map of the surface is obtained by a distance sensor. ### Claim 11 (depends on 8) The method of claim 8, further comprising: determining a fitting plane based on position information and depth information of each pixel in the depth map; determining the flatness as a deviation of the depth map from the fitting plane; and determining the tilt of the depth map based on an angle between a normal vector normal to the fitting plane and a gravity vector extending in a direction of gravity. ### Claim 12 (depends on 1) The method of claim 1, wherein determining the flatness or the tilt of the surface based on the data measured by the sensors comprises: controlling the movable object to capture an image; determining whether the image is captured when the movable object is at a horizontal state; in response to determining that the image is not captured when the movable object is at the horizontal state, applying a perspective projection to the image; and determining the flatness or the tilt of the surface based on the perspective projection to the image. ### Claim 13 (depends on 1) The method of claim 1, wherein: determining the flatness or the tilt of the surface based on the data measured by one or more the sensors includes: generating a depth map including first depth information of first pixels of at least one image of the surface, each first pixels corresponding to a portion of the surface, and the first depth information including a height of a corresponding portion of the surface; and controlling the movable object to land based on the flatness or the tilt further includes: in response to the movable object descending toward the surface, controlling the movable object to capture at least one new image of the surface; updating the depth map to include second depth information of second pixels of the at least one new image of the surface, each second pixels corresponding to a portion of the surface; and determining, based on the updated depth map, whether the surface satisfies a landing condition as the movable object descends. ### Claim 14 (depends on 1) The method of claim 1, further comprising: in response to the movable object descending toward the surface, monitoring an elevation of the movable object; and in response to detecting a disruptively change of the elevation of the movable object, determining that the surface is not suitable for landing, and controlling the movable object to hover at a current altitude or controlling the movable object to ascend to an altitude higher than the current altitude, the disruptively change of the elevation of the movable object including a situation that a person or an object is on the surface. ### Claim 15 (depends on 1) The method of claim 1, wherein the one or more sensors include at least one of a single camera, a pair of cameras, an ultrasonic sensor, an IMU, a laser rangefinder, or a visual sensor. ### Claim 16 (depends on 1) The method of claim 1, wherein controlling the movable object to land based on the flatness or the tilt further comprises: detects characteristics of a terrain of a landing surface and identifies a spot or area with a flat or smooth surface suitable for landing. ### Claim 17 An aerial vehicle apparatus, comprising: one or more processors; and one or more memories storing one or more computer programs that, when executed by the one or more processors, cause the one or more processors to: receive a request to land the aerial vehicle apparatus on a surface; determine flatness or tilt of the surface based on data measured by one or more sensors; and control the aerial vehicle apparatus to land based on the flatness or the tilt of the surface, including: in response to determining that the surface is not flat or is tilted, controlling the aerial vehicle apparatus to move to a location corresponding to another surface or controlling the aerial vehicle apparatus to hover. ### Claim 18 (depends on 17) The aerial vehicle apparatus of claim 17, the one or more computer programs, when executed by the one or more processors, further cause the one or more processors to: control the aerial vehicle apparatus to descend to a predetermined height before obtaining the data measured by the sensors. ### Claim 19 (depends on 17) The aerial vehicle apparatus of claim 17, wherein determining flatness or the tilt of the surface based on the data measured by the one or more sensors comprises: obtaining a depth map of the surface based on the data measured by the one or more sensors, each pixel of the depth map corresponding to a portion of the surface; and determining the flatness or the tilt of the depth map. ### Claim 20 A non-transitory computer-readable medium storing instructions that, when executed, cause one or more processors to perform a method for controlling landing of a movable object, the method comprising: receiving a request to land the movable object on a surface; determining flatness or tilt of the surface based on data measured by sensors; and controlling the movable object to land based on the flatness or the tilt, including: in response to determining that the surface is not flat or is tilted, controlling the movable object to move to a location corresponding to another surface or controlling the movable object to hover.