# US 2023/0274653 A1 · Techniques for sharing mapping data between an unmanned aerial vehicle and a ground vehicle

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0274653 A1
- Document ID: US20230274653A1
- Status: Published application
- Kind: A1
- Application number: 18/127,727
- Filed: 2023-03-29
- Published: 2023-08-31
- Earliest claimed: 2019-03-08
- Assignees: Sz Dji Technology Co., Ltd.
- Applicants: Sz Dji Technology Co., Ltd.
- Inventors: Mingyu WANG
- Web page: https://patentgrep.com/patent/US20230274653A1

## CPC

- G08G 5/55: for a single aircraft
- G01C 21/3815: {Road data}
- G01C 21/3848: {Data obtained from both position sensors and additional sensors}
- G01C 21/3852: {Data derived from aerial or satellite images}
- G01S 17/89: for mapping or imaging
- G05D 1/0094: {involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target}
- G05D 1/024: {in combination with a laser (lasers per se H01S)}
- G05D 1/0274: {using mapping information stored in a memory device (navigation using map-matching G01C21/30)}
- G05D 1/028: {using a RF signal}
- G06F 18/251: {of input or preprocessed data}
- G06V 10/56: relating to colour
- G06V 20/13: Satellite images
- G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- G08G 1/012: {from other sources than vehicle or roadside beacons, e.g. mobile networks}
- G08G 1/0125: {Traffic data processing}
- G08G 1/0133: {for classifying traffic situation}
- G08G 1/04: using optical or ultrasonic detectors
- G08G 1/096716: {where the received information does not generate an automatic action on the vehicle control}
- G08G 1/096741: {where the source of the transmitted information selects which information to transmit to each vehicle}
- G08G 1/09675: {where a selection from the received information takes place in the vehicle}
- G08G 1/096791: {where the origin of the information is another vehicle}
- G08G 5/26: Transmission of traffic-related information between aircraft and ground stations
- G08G 5/57: for unmanned aircraft
- G08G 5/723: {from the aircraft}
- G08G 5/74: for monitoring terrain
- G08G 7/02: Anti-collision systems
- H04W 4/024: Guidance services
- H04W 4/025: {using location based information parameters}
- G01S 19/14: specially adapted for specific applications
- G01S 19/47: the supplementary measurement being an inertial measurement, e.g. tightly coupled inertial
- G08G 1/161: {Decentralised systems, e.g. inter-vehicle communication}
- G08G 1/166: {for active traffic, e.g. moving vehicles, pedestrians, bikes}
- H04W 4/46: for vehicle-to-vehicle communication [V2V]
- G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93})
- G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S)
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72)

## Abstract

Techniques are disclosed for sharing sensor information between multiple vehicles. A system for sharing sensor information between multiple vehicles, can include an aerial vehicle including a first computing device and first scanning sensor, and a ground vehicle including a second computing device and second scanning. The aerial vehicle can use the first scanning sensor to obtain first scanning data and transmit the first scanning data to the second computing device. The ground vehicle can receive the first scanning data from the first computing device, obtain second scanning data from the second scanning sensor, identify an overlapping portion of the first scanning data and the second scanning data based on at least one reference object in the scanning data, and execute a navigation control command based on one or more roadway objects identified in the overlapping portion of the first scanning data and the second scanning data.

## Claims

### Claim 1

A system for generating a map based on sensor information from multiple vehicles, comprising:
an aerial vehicle including a first computing device;
a first scanning sensor coupled to the aerial vehicle;
a ground vehicle including a second computing device:
a second scanning sensor coupled to the ground vehicle;
the first computing device including at least one processor and a scanning manager, the scanning manager including first instructions which, when executed by the processor, cause the scanning manager to:
obtain first scanning data from the first scanning sensor from an overhead perspective; and
transmit the first scanning data to the second computing device; and
the second computing device including at least one processor and a detection manager, the detection manager including second instructions which, when executed by the processor, cause the detection manager to:
receive the first scanning data from the first computing device;
identify one or more roadway objects in the first scanning data;
map one or more positions associated with the one or more roadway objects in the first scanning data to second scanning data obtained from a second scanning sensor from a forward perspective, and
execute a navigation control command based on the one or more roadway objects mapped to the second scanning data.

### Claim 2 (depends on 1)

The system of claim 1, wherein to map the one or more positions associated with the one or more roadway objects in the first scanning data to second scanning data obtained from a second scanning sensor from a forward perspective, the second instructions, when executed, further cause the detection manager to:
transform the second scanning data from the forward perspective to the overhead perspective to generate transformed second scanning data;
calibrate the transformed second scanning data based on a reference object represented in the transformed second scanning data and the first scanning data,
identify the one or more positions associated with the one or more roadway objects in the transformed second scanning data; and
convert the one or more positions associated with the one or more roadway objects in the transformed second scanning data to the forward perspective by performing a reverse perspective transformation.

### Claim 3 (depends on 1)

The system of claim 1, wherein the first scanning sensor includes an imaging sensor and the second scanning sensor includes an imaging sensor.

### Claim 4 (depends on 3)

The system of claim 3, wherein the first scanning data includes first mapping data generated based on image data collected by the first scanning sensor and wherein the second scanning data includes second mapping data generated based on image data collected by the second scanning sensor.

### Claim 5 (depends on 4)

The system of claim 4, wherein the second instructions, when executed, further cause the detection manager to: combine the first mapping data with the second mapping data to increase a coverage area of a local map maintained by the ground vehicle.

### Claim 6 (depends on 1)

The system of claim 1, wherein the first scanning sensor includes a first LiDAR sensor and the second scanning sensor includes a second LiDAR sensor.

### Claim 7 (depends on 6)

The system of claim 6, wherein the first scanning data includes first mapping data generated based on point cloud data collected by the first scanning sensor and wherein the second scanning data includes second mapping data generated based on point cloud data collected by the second scanning sensor.

### Claim 8 (depends on 7)

The system of claim 7, wherein the second instructions, when executed, further cause the detection manager to combine the first mapping data with the second mapping data to increase a coverage area of a local map maintained by the ground vehicle.

### Claim 9 (depends on 1)

The system of claim 1, further comprising: one or more ground vehicles in communication with the aerial vehicle, the one or more ground vehicles to each receive the first scanning data from the aerial vehicle.

### Claim 10 (depends on 1)

The system of claim 1, further comprising: one or more aerial vehicles in communication with the ground vehicle, wherein each aerial vehicle from the one or more aerial vehicles obtains third scanning data and transmits the third scanning data to the ground vehicle.

### Claim 11

A system for generating a map based on sensor information from multiple vehicles, comprising:
an aerial vehicle including a first computing device,
a first scanning sensor coupled to the aerial vehicle;
a ground vehicle including a second computing device;
a second scanning sensor coupled to the ground vehicle;
the first computing device including at least one processor and a scanning manager, the scanning manager including first instructions which, when executed by the processor, cause the scanning manager to:
obtain first scanning data from the first scanning sensor from an overhead perspective; and
transmit the first scanning data to the second computing device; and
the second computing device including at least one processor and a detection manager, the detection manager including second instructions which, when executed by the processor, cause the detection manager to:
receive the first scanning data from the first computing device;
generate a local map based at least in part on the first scanning data and
execute a navigation control command based at least in part on the local map.

### Claim 12 (depends on 11)

The system of claim 11, wherein the second instructions, when executed, further cause the detection manager to:
obtain second scanning data from the second scanning sensor, wherein the second scanning data is obtained from the second scanning sensor from a forward perspective.

### Claim 13 (depends on 12)

The system of claim 12, wherein to generate a local map based at least in part on the first scanning data, the second instructions, when executed, further cause the detection manager to:
transform the second scanning data from the forward perspective to the overhead perspective to obtain transformed second scanning data based at least in part on the first scanning data; and
generate the local map based on the transformed second scanning data.

### Claim 14 (depends on 12)

The system of claim 12, wherein to generate a local map based at least in part on the first scanning data, the second instructions, when executed, further cause the detection manager to:
identify a portion of the first scanning data that corresponds to a portion of the second scanning data, wherein the portion of the first scanning data and the portion of the second scanning data include a representation of a same area of a roadway environment; and
generate the local map based on the identified portion of the first scanning data.

### Claim 15 (depends on 11)

The system of claim 11, wherein the first scanning sensor includes an imaging sensor and the second scanning sensor includes an imaging sensor.

### Claim 16 (depends on 15)

The system of claim 15, wherein the first scanning data includes first mapping data generated based on image data collected by the first scanning sensor and wherein the second scanning data includes second mapping data generated based on image data collected by the second scanning sensor.

### Claim 17 (depends on 11)

The system of claim 11, wherein the first scanning sensor includes a first LiDAR sensor and the second scanning sensor includes a second LiDAR sensor.

### Claim 18 (depends on 17)

The system of claim 17, wherein the first scanning data includes first mapping data generated based on point cloud data collected by the first scanning sensor and wherein the second scanning data includes second mapping data generated based on point cloud data collected by the second scanning sensor.

### Claim 19 (depends on 11)

The system of claim 11, further comprising: one or more ground vehicles in communication with the aerial vehicle, the one or more ground vehicles to each receive the first scanning data from the aerial vehicle.

### Claim 20 (depends on 11)

The system of claim 11, further comprising: one or more aerial vehicles in communication with the ground vehicle, wherein each aerial vehicle from the one or more aerial vehicles obtains third scanning data and transmits the third scanning data to the ground vehicle.
