# US 12,223,698 B2 · Method for searching a path by using a three-dimensional reconstructed map

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,223,698 B2
- Document ID: US12223698B2
- Status: Granted
- Kind: B2
- Application number: 17/825,503
- Filed: 2022-05-26
- Published: 2023-08-24
- Granted: 2025-02-11
- Earliest claimed: 2022-05-26
- Assignees: National Yang Ming Chiao Tung University
- Applicants: National Yang Ming Chiao Tung University
- Inventors: Mang Ou-Yang; Yung-Jhe Yan; Ming-Da Jiang; Ta-Fu Hsu; Shao-Chun Yeh; Kun-Hsiang Chen; Tzung-Cheng Chen
- Web page: https://patentgrep.com/patent/US12223698B2

## CPC

- G06V 10/803: {of input or preprocessed data}
- B64C 39/024: {of the remote controlled vehicle type, i.e. RPV}
- G01C 21/005: {with correlation of navigation data from several sources, e.g. map or contour matching (G01C21/30 takes precedence)}
- G01C 21/3826: {Terrain data}
- G05D 1/102: {specially adapted for vertical take-off of aircraft}
- G05D 1/106: {Change initiated in response to external conditions, e.g. avoidance of elevated terrain or of no-fly zones}
- G05D 1/606: Compensating for or utilising external environmental conditions, e.g. wind or water currents (station keeping G05D1/611)
- G06V 10/56: relating to colour
- G06V 10/762: using clustering, e.g. of similar faces in social networks
- G06V 10/80: Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level (multimodal speaker identification or verification G10L17/10)
- G06V 10/82: using neural networks
- G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- G08G 5/30: Flight plan management
- G08G 5/55: for a single aircraft
- G08G 5/57: for unmanned aircraft
- B64U 2201/10: autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS]
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- B64C: AEROPLANES; HELICOPTERS
- G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S)
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93})
- B64U: UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR

## Abstract

A method for searching a path by using a 3D reconstructed map includes: receiving 3D point-cloud map information and 3D material map information; clustering the 3D point-cloud map information with a clustering algorithm to obtain clustering information, and identifying material attributes of objects in the 3D point-cloud map information with a material neural network model to obtain material attribute information; fusing the those map information based on their coordinate information, thereby outputting fused map information; identifying obstacle areas and non-obstacle areas in the fused map information based on an obstacle neural network model, the clustering information, and the material attribute information; and generating 3D path information according to the non-obstacle areas. Since the 3D path information is generated based on those map information, the obstacle areas and flight spaces are effectively determined to generate an accurate flight path.

## Claims

### Claim 1

A method for searching a path by using a three-dimensional reconstructed map comprising:
using a point-cloud map-retrieving device to emit lidar signals to environment, detect distance data points according to the lidar signals, and combine the distance data points into three-dimensional point-cloud map information according to actual coordinates of the distance data points;
using a processor to perform the following steps:
receiving the three-dimensional point-cloud map information and three-dimensional material map information;
clustering the three-dimensional point-cloud map information with a clustering algorithm to obtain clustering information, and identifying material attributes of objects in the three-dimensional point-cloud map information with a material neural network model to obtain material attribute information;
retrieving coordinate information of the three-dimensional point-cloud map information and the three-dimensional material map information, and fusing the three-dimensional point-cloud map information and the three-dimensional material map information based on the coordinate information, thereby outputting fused map information;
identifying obstacle areas and non-obstacle areas in the fused map information based on an obstacle neural network model, the clustering information, and the material attribute information;
generating three-dimensional path information according to the non-obstacle areas, wherein the three-dimensional path information comprises path points connected with each other, and the path points keep a distance where an obstacle-avoidance circle is formed from the obstacle areas, the distance is used as the radius of the obstacle-avoidance circle; and
inputting wind information and determining whether the wind information is greater than a wind-level threshold or determining whether a headwind occurs based on the wind information, wherein a wind sensor is coupled to the processor generating the wind information:
if yes, decreasing the radius of the obstacle-avoidance circle and a position in a z axis of the three-dimensional path information to generate first three-dimensional path information, wherein the position is unmanned aerial vehicle's position in the z axis of the three-dimensional path information; and
if no, increasing the radius of the obstacle-avoidance circle to generate second three-dimensional path information.

### Claim 2 (depends on 1)

The method for searching a path by using a three-dimensional reconstructed map according to claim 1, wherein the radius of the obstacle-avoidance circle is decreased by at most 0.7 m, and the radius of the obstacle-avoidance circle is increased by at most 0.5 m.

### Claim 3 (depends on 1)

The method for searching a path by using a three-dimensional reconstructed map according to claim 1, wherein the positions in the z axis of the three-dimensional path information are decreased by 0.5˜2 m.

### Claim 4 (depends on 1)

The method for searching a path by using a three-dimensional reconstructed map according to claim 1, wherein the non-obstacle area is at least 0.5 m from the obstacle area.

### Claim 5 (depends on 1)

The method for searching a path by using a three-dimensional reconstructed map according to claim 1, wherein the material neural network model is pre-trained using historical data of material attributes of different objects.

### Claim 6 (depends on 5)

The method for searching a path by using a three-dimensional reconstructed map according to claim 5, wherein the material attributes are color attributes.

### Claim 7 (depends on 1)

The method for searching a path by using a three-dimensional reconstructed map according to claim 1, wherein the obstacle neural network model is pre-trained using historical data of material attribute information and clustering information of different obstacle areas.

### Claim 8 (depends on 1)

The method for searching a path by using a three-dimensional reconstructed map according to claim 1, wherein the clustering algorithm is density-based spatial clustering of applications with noise (DBSCAN).

### Claim 9 (depends on 1)

The method for searching a path by using a three-dimensional reconstructed map according to claim 1, wherein the three-dimensional point-cloud map information is three-dimensional lidar point-cloud map information, and the three-dimensional material map information is three-dimensional image information.