# US 12,015,757 B2 · Obstacle detection method and apparatus and unmanned aerial vehicle

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,015,757 B2
- Document ID: US12015757B2
- Status: Granted
- Kind: B2
- Application number: 17/310,718
- Filed: 2020-02-26
- Published: 2022-06-16
- Granted: 2024-06-18
- Earliest claimed: 2019-02-28
- Assignees: Autel Robotics Co., Ltd.
- Applicants: Autel Robotics Co., Ltd.
- Inventors: Xin Zheng
- Web page: https://patentgrep.com/patent/US12015757B2

## CPC

- G06V 20/17: taken from planes or by drones
- H04N 13/254: in combination with electromagnetic radiation sources for illuminating objects
- B64C 39/024: {of the remote controlled vehicle type, i.e. RPV}
- B64D 45/08: optical
- B64D 47/00: Equipment not otherwise provided for
- G01J 1/4204: {with determination of ambient light (solar light G01J2001/4266)}
- G05D 1/101: {specially adapted for aircraft}
- G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI]
- G06V 10/80: Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level (multimodal speaker identification or verification G10L17/10)
- G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- H04N 13/239: using two two-dimensional [2D] image sensors having a relative position equal to or related to the interocular distance (H04N13/243 takes precedence)
- H04N 13/296: Synchronisation thereof; Control thereof
- B64U 10/13: Flying platforms
- B64U 2101/00: UAVs specially adapted for particular uses or applications
- B64U 2101/30: for imaging, photography or videography
- H04N 2013/0081: {Depth or disparity estimation from stereoscopic image signals}
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- B64C: AEROPLANES; HELICOPTERS
- B64D: EQUIPMENT FOR FITTING IN OR TO AIRCRAFT; FLIGHT SUITS; PARACHUTES; ARRANGEMENT OR MOUNTING OF POWER PLANTS OR PROPULSION TRANSMISSIONS IN AIRCRAFT
- G01J: MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY (light sources F21, H01J, H01K, H05B; investigating properties of materials by optical means G01N)
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- B64U: UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR

## Abstract

Embodiments of the present invention relate to an obstacle detection method and apparatus and an unmanned aerial vehicle. The unmanned aerial vehicle includes a binocular photographing component and a laser texture component. The method includes: determining to start the laser texture component; starting the laser texture component, to emit a laser texture; obtaining a binocular view that is collected by the binocular photographing component and that includes the laser texture, and setting the binocular view that includes the laser texture as a target binocular view; and performing obstacle detection based on the target binocular view. In the above technical solutions, according to the embodiments of the present invention, precision of binocular stereo matching can be improved without changing an original binocular matching algorithm and structure, thereby improving precision of obstacle detection. In addition, the unmanned aerial vehicle can perform binocular sensing while flying at night.

## Claims

### Claim 1

An obstacle detection method, applied to an unmanned aerial vehicle, wherein the unmanned aerial vehicle comprises a binocular photographing component and a laser texture component, and the method comprises:
determining to start the laser texture component;
starting the laser texture component, to emit a laser texture;
obtaining a binocular view that is collected by the binocular photographing component and that comprises the laser texture, and setting the binocular view that comprises the laser texture as a target binocular view; and
performing obstacle detection based on the target binocular view;
wherein the determining to start the laser texture component comprises:
determining that a detection zone of the laser texture component has an obstacle;
wherein the determining that a detection zone of the laser texture component has an obstacle comprises:
obtaining an initial binocular view currently collected by the binocular photographing component:
determining a distance between the unmanned aerial vehicle and an obstacle closest to the unmanned aerial vehicle based on the initial binocular view; and
when the distance is less than or equal to a detection distance of the laser texture component determining that the detection zone of the laser texture component has an obstacle;
wherein the determining to start the laser texture component further comprises:
determining that a photographing scene of the binocular photographing component is a weak texture scene;
wherein the determining that a photographing scene of the binocular photographing component is a weak texture scene comprises:
obtaining an initial view of at least one initial binocular views currently collected by the binocular photographing component;
performing a gradient operation on the initial view, to obtain a gradient value of each pixel in the initial view;
collecting statistics on a quantity N of pixels whose gradient values are less than or equal to a gradient threshold; and
when the quantity N is less than or equal to a quantity threshold Nth, determining that the photographing scene of the binocular photographing component is the weak texture scene.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the determining to start the laser texture component further comprises:
determining that a photographing scene of the binocular photographing component is a repeated texture scene.

### Claim 3 (depends on 2)

The method according to claim 2, wherein the determining that a photographing scene of the binocular photographing component is a repeated texture scene comprises:
obtaining an initial binocular view currently collected by the binocular photographing component;
performing stereo matching on the initial binocular view, and obtaining a smallest cost value C1i and a second smallest cost value C2i that correspond to each matching block, wherein i indicates an i-th matching block; and
determining that the photographing scene of the binocular photographing component is the repeated texture scene if there is a matching block i that satisfies the following formula:
C1i/C2i>K, wherein
A value range of K is: 0.5≤K<1.

### Claim 4 (depends on 1)

The method according to claim 1, wherein the determining that luminance of an environment of the unmanned aerial vehicle is less than a preset value comprises:
an initial view of initial binocular views currently collected by the binocular photographing component; and
determining that the luminance of the environment of the unmanned aerial vehicle is less than the preset value according to an average gray scale value of pixels in the initial view.

### Claim 5

An obstacle detection apparatus, applicable to an unmanned aerial vehicle, wherein the unmanned aerial vehicle comprises a binocular photographing component and a laser texture component, and the obstacle detection apparatus comprises:
a memory, configured to store a computer-executable obstacle detection program; and
a processor, configured to invoke the computer-executable obstacle detection program to implement:
determine to start the laser texture component;
start the laser texture component, to emit a laser texture;
obtain a binocular view that is collected by the binocular photographing component and that comprises the laser texture, and setting the binocular view that comprises the laser texture as a target binocular view; and
perform obstacle detection based on the target binocular view, the processor is further configured to determine that a detection zone of the laser texture component has an obstacle;
obtain an initial binocular view currently collected by the binocular photographing component;
determine a distance between the unmanned aerial vehicle and an obstacle closest to the unmanned aerial vehicle based on the initial binocular view; and
when the distance is less than or equal to a detection distance of the laser texture component, determine that the detection zone of the laser texture component has an obstacle;
determine that a photographing scene of the binocular photographing component is a weak texture scene;
obtain an initial view of initial binocular views currently collected by the binocular component;
perform a gradient operation on the initial view, to obtain a gradient value of each pixel in the initial view;
collect statistics on a quantity N of pixels whose gradient values are less than or equal to a gradient threshold; and
when the quantity N is less than or equal to a quantity threshold Nth, determine that the photographing scene of the binocular photographing component is the weak texture scene.

### Claim 6 (depends on 5)

The apparatus according to claim 5, wherein
the processor is further configured to determine that a photographing scene of the binocular photographing component is a repeated texture scene.

### Claim 7 (depends on 6)

The apparatus according to claim 6, wherein the processor is further configured to:
obtain an initial binocular view currently collected by the binocular photographing component;
perform stereo matching on the initial binocular view, and obtain a smallest cost value C1i and a second smallest cost value C2i that correspond to each matching block, wherein i indicates an ith matching block; and
determine that the photographing scene of the binocular photographing component is the repeated texture scene if there is a matching block i that satisfies the following formula:
C1i/C2i>K, wherein
a value range of K is: 0.5≤K<1.

### Claim 8 (depends on 5)

The apparatus according to claim 5, wherein the processor is further configured to:
obtain an initial view of initial binocular views currently collected by the binocular photographing component; and
determine that the luminance of the environment of the unmanned aerial vehicle is less than the preset value according to an average grayscale value of pixels in the initial view.

### Claim 9

An unmanned aerial vehicle, comprising:
a vehicle body;
a vehicle arm, connected to the vehicle body;
a power apparatus, disposed on the vehicle arm;
a binocular photographing component, disposed on the vehicle body, and configured to collect a binocular view in a movement direction of the unmanned aerial vehicle;
a laser texture component, disposed on the vehicle body, and configured to emit a laser texture sensible by the binocular photographing component;
a processor, disposed inside the vehicle body, and being in communication connection with each of the binocular photographing component and the laser texture component; and
a memory, being in communication connection with the processor, wherein
the memory stores instructions executable by the processor, and the instructions are executed by the processor to enable the processor to execute:
determining to start the laser texture component;
starting the laser texture component; to emit a laser texture;
obtaining a binocular view that is collected by the binocular photographing component and that comprises the laser texture, and setting the binocular view that comprises the laser texture as a target binocular view; and
performing obstacle detection based on the target binocular view;
wherein starting the laser texture component to emit a laser texture further comprises:
determining that a detection zone of the laser texture component has an obstacle;
wherein the determining that a detection zone of the laser texture component has an obstacle comprises:
obtaining an initial binocular view currently collected by the binocular photographing component;
determining a distance between the unmanned aerial vehicle and an obstacle closest to the unmanned aerial vehicle based on the initial binocular view; and
when the distance is less than or equal to a detection distance of the laser texture component, determining that the detection zone of the laser texture component has an obstacle;
wherein the determining to start the laser texture component further comprises:
determining that a photographing scene of the binocular photographing component is a weak texture scene;
wherein the determining that a photographing scene of the binocular photographing component is a weak texture scene comprises:
obtaining an initial view of at least one initial binocular views currently collected by the binocular photographing component;
performing a gradient operation on the initial view, to obtain a gradient value of each pixel in the initial view;
collecting statistics on a quantity N of pixels whose gradient values are less than or equal to a gradient threshold; and
when the quantity N is less than or equal to a quantity threshold Nth, determining that the photographing scene of the binocular photographing component is the weak texture scene.

### Claim 10

A non-transitory computer-readable storage medium, storing computer-executable instructions, the computer-executable instructions being configured to cause an unmanned aerial vehicle to execute:
determining to start the laser texture component;
starting the laser texture component, to emit a laser texture;
obtaining a binocular view that is collected by the binocular photographing component and that comprises the laser texture, and setting the binocular view that comprises the laser texture as a target binocular view; and
performing obstacle detection based on the target binocular view wherein starting the laser texture component to emit a laser texture further comprises:
determining that a detection zone of the laser texture component has an obstacle;
wherein the determining that a detection zone of the laser texture component has an obstacle comprises:
obtaining an initial binocular view currently collected by the binocular photographing component;
determining a distance between the unmanned aerial vehicle and an obstacle closest to the unmanned aerial vehicle based on the initial binocular view and
when the distance is less than or equal to a detection distance of the laser texture component, determining that the detection zone of the laser texture component has an obstacle;
wherein the determining to start the laser texture component further comprises:
determining that a photographing scene of the binocular photographing component is a weak texture scene;
wherein the determining that a photographing scene of the binocular photographing component is a weak texture scene comprises:
obtaining an initial view of at least one initial binocular views currently collected by the binocular photographing component;
performing a gradient operation on the initial view, to obtain a gradient value of each pixel in the initial view;
collecting statistics on a quantity N of pixels whose gradient values are less than or equal to a gradient threshold; and
when the quantity N is less than or equal to a quantity threshold Nth, determining that the photographing scene of the binocular photographing component is the weak texture scene.