# US 2021/0064024 A1 · Scanning environments and tracking unmanned aerial vehicles

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0064024 A1
- Document ID: US20210064024A1
- Status: Published application
- Kind: A1
- Application number: 16/849,924
- Filed: 2020-04-15
- Published: 2021-03-04
- Earliest claimed: 2015-03-03
- Assignees: PreNav, Inc.
- Applicants: PreNav, Inc.
- Inventors: Asa Hammond; Nathan Schuett; Naimisaranya Das Busek
- Web page: https://patentgrep.com/patent/US20210064024A1

## CPC

- G05D 1/102: {specially adapted for vertical take-off of aircraft}
- B64U 10/13: Flying platforms
- G01C 21/3415: {Dynamic re-routing, e.g. recalculating the route when the user deviates from calculated route or after detecting real-time traffic data or accidents}
- G01S 7/4813: {Housing arrangements}
- G01S 17/66: Tracking systems using electromagnetic waves other than radio waves
- G01S 17/86: Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
- G01S 17/87: Combinations of systems using electromagnetic waves other than radio waves
- G01S 17/89: for mapping or imaging
- G01S 17/933: of aircraft or spacecraft
- G05D 1/0033: {by having the operator tracking the vehicle either by direct line of sight or via one or more cameras located remotely from the vehicle}
- G05D 1/0044: {by providing the operator with a computer generated representation of the environment of the vehicle, e.g. virtual reality, maps (maps used for automatic navigation G05D1/0274; flight directors G01C23/005)}
- G05D 1/0094: {involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target}
- G06F 3/0484: for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F 18/24: Classification techniques
- G06T 7/0004: {Industrial image inspection}
- G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51)
- G06T 7/60: Analysis of geometric attributes
- G06T 17/05: Geographic models
- G06V 20/653: {by matching three-dimensional models, e.g. conformal mapping of Riemann surfaces}
- G08G 5/55: for a single aircraft
- G08G 5/57: for unmanned aircraft
- G08G 5/80: Anti-collision systems
- H04N 13/128: Adjusting depth or disparity
- B64U 2101/30: for imaging, photography or videography
- B64U 2201/20: Remote controls
- F03D 17/00: Monitoring or testing of wind motors, e.g. diagnostics (testing during commissioning of wind motors F03D13/30)
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2207/10032: Satellite or aerial image; Remote sensing
- G06T 2207/20221: Image fusion; Image merging
- G06T 2207/30181: Earth observation
- G06T 2207/30241: Trajectory
- G06T 2207/30252: Vehicle exterior; Vicinity of vehicle
- H04N 2013/0081: {Depth or disparity estimation from stereoscopic image signals}
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- B64U: UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
- G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S)
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93})
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- F03D: WIND MOTORS

## Abstract

Systems and methods for scanning environments and tracking unmanned aerial vehicles within the scanned environments are disclosed. A method in accordance with a particular embodiment includes using a rangefinder off-board an unmanned air vehicle (UAV) to identify points in a region. The method can further include forming a computer-based map of the region with the points and using the rangefinder and a camera to locate the UAV as it moves in the region. The location of the UAV can be compared with locations on the computer-based map and, based upon the comparison, the method can include transmitting guidance information to the UAV. In a further particular embodiment, two-dimensional imaging data is used in addition to the rangefinder data to provide color information to points in the region.

## Claims

### Claim at position 1

-22. (canceled)

### Claim 23

A non-transitory computer-readable storage medium storing instructions that, if executed by a computing system having a memory and a processor, cause the computing system to perform a method, the method comprising:
providing information characterizing a visual representation of a three-dimensional model of the environment, the environment including a plurality of objects;
receiving input related to a proposed flight path for an unmanned aerial vehicle through the environment, wherein the input includes one or more regions of interest;
receiving a predicted trajectory of the unmanned aerial vehicle through the environment, with the predicted trajectory calculated based at least in part on the received input, one or more characteristics for pointing a sensor of the unmanned aerial vehicle toward the one or more regions of interest, and one or more flight characteristics of the unmanned aerial vehicle;
determining whether the predicted trajectory of the unmanned aerial vehicle through the environment is free of collisions;
if the predicted trajectory is determined to be not free of collisions, revising the predicted trajectory to avoid a collision; and
providing at least part of the revised trajectory to the unmanned aerial vehicle for a flight.

### Claim 24 (depends on 23)

The computer-readable medium of claim 23 wherein the information is provided for presentation to a user, and wherein the input is received from the user.

### Claim 25 (depends on 24)

The computer-readable medium of claim 24, wherein the input is a first input, and wherein the method further comprises:
based on a second input from the user, shifting a viewpoint from which the user views the visual representation of the three-dimensional model of the environment.

### Claim 26 (depends on 23)

The computer-readable medium of claim 23, wherein in the method further comprises:
generating the three-dimensional model of the environment at least in part by scanning the environment with at least one of a camera or a rangefinder.

### Claim 27 (depends on 23)

The computer-readable medium of claim 23, wherein in the method further comprises:
generating the three-dimensional model of the environment based at least in part on information collected from at least one of a camera, a rangefinder, a tilt encoder, and a pan encoder, at least in part by:
capturing images of the environment;
generating depth data for objects within the captured images at least in part by determining, estimating, or both determining and estimating the distance from the rangefinder to each of a plurality of the objects; and
transforming the generated depth data to Cartesian coordinates based at least in part on information collected from the tilt encoder and the pan encoder.

### Claim 28 (depends on 23)

The computer-readable medium of claim 23, wherein in the method further comprises:
importing the three-dimensional model of the environment.

### Claim 29 (depends on 23)

The computer-readable medium of claim 23, wherein in the method further comprises:
generating the three-dimensional model of the environment at least in part by extracting spatial information from images captured by at least one of the unmanned aerial vehicle or another unmanned aerial vehicle.

### Claim 30 (depends on 23)

The computer-readable medium of claim 23, wherein the received input comprises:
for an individual region of interest, a distance away from the region of interest that the unmanned aerial vehicle is to fly.

### Claim 31 (depends on 23)

The computer-readable medium of claim 23, wherein determining whether the predicted trajectory of the unmanned aerial vehicle through the environment is free of collisions comprises:
comparing the three-dimensional model of the environment to the predicted trajectory of the unmanned aerial vehicle through the environment to determine whether the unmanned aerial vehicle would make contact with an object in the environment.

### Claim 32 (depends on 23)

The computer-readable medium of claim 23, wherein in the method further comprises:
if the predicted trajectory is determined to be not free of collisions, providing an alert.

### Claim 33 (depends on 32)

The computer-readable medium of claim 32, wherein the alert includes at least one of:
an audible signal;
a flashing light; or
displaying an interfering portion of the predicted trajectory in a different manner than a non-interfering portion of the predicted trajectory.

### Claim 34 (depends on 23)

The computer-readable medium of claim 23, wherein the received input includes at least one of a proximity constraint and a speed for the unmanned aerial vehicle.

### Claim 35 (depends on 23)

The computer-readable medium of claim 23, wherein in the method further comprises:
causing the unmanned aerial vehicle to begin a flight in accordance with the predicted trajectory of the unmanned aerial vehicle through the environment; and:
tracking the flight of the unmanned aerial vehicle through the environment with a first tracker.

### Claim 36 (depends on 35)

The computer-readable medium of claim 35, wherein the method further comprises:
handing off responsibility for tracking the unmanned aerial vehicle from the first scanner to a second tracker while the unmanned aerial vehicle is in flight.

### Claim 37 (depends on 35)

The computer-readable medium of claim 35, wherein tracking the flight of the unmanned aerial vehicle through the environment comprises combining corner cube interferometry with output from a laser-based position sensing device to generate position estimates for the unmanned aerial vehicle.

### Claim 38 (depends on 35)

The computer-readable medium of claim 35, wherein the method further comprises:
while tracking the flight of the unmanned aerial vehicle through the environment, comparing the flight of the unmanned aerial vehicle through the environment to the predicted trajectory of the unmanned aerial vehicle through the environment.

### Claim 39 (depends on 38)

The computer-readable medium of claim 38, wherein the method further comprises:
adjusting the flight of the unmanned aerial vehicle in response to determining, based on the comparing, that the unmanned aerial vehicle has deviated from the predicted trajectory of the unmanned aerial vehicle through the environment.
