# US 2020/0402410 A1 · Unmanned Aerial Vehicle Visual Line Of Sight Control

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0402410 A1
- Document ID: US20200402410A1
- Status: Published application
- Kind: A1
- Application number: 17/008,344
- Filed: 2020-08-31
- Published: 2020-12-24
- Earliest claimed: 2016-02-08
- Assignees: Skydio, Inc.
- Applicants: Skydio, Inc.
- Inventors: Dana Livonia Contreras; Lucas Palage Doyle; Justin Eugene Kuehn; Volkan Gurel; Eric David Johnson; Edward Dale Steakley
- Web page: https://patentgrep.com/patent/US20200402410A1

## CPC

- G08G 5/55: for a single aircraft
- B64C 39/024: {of the remote controlled vehicle type, i.e. RPV}
- B64D 47/08: Arrangements of cameras
- G05D 1/0033: {by having the operator tracking the vehicle either by direct line of sight or via one or more cameras located remotely from the vehicle}
- G05D 1/0094: {involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target}
- G05D 1/0274: {using mapping information stored in a memory device (navigation using map-matching G01C21/30)}
- G05D 1/2235: {involving the operator tracking the vehicle by direct line of sight}
- G05D 1/229: Command input data, e.g. waypoints
- G05D 1/246: using environment maps, e.g. simultaneous localisation and mapping [SLAM]
- G05D 1/80: Arrangements for reacting to or preventing system or operator failure (handing over between remote control and on-board control, or handing over between remote control arrangements G05D1/227)
- G06T 5/70: Denoising; Smoothing
- G06V 20/64: Three-dimensional [3D] objects
- G08G 5/57: for unmanned aircraft
- G08G 5/59: in accordance with predefined flight zones, e.g. to avoid prohibited zones
- H04B 7/18506: {Communications with or from aircraft, i.e. aeronautical mobile service}
- B64U 10/13: Flying platforms
- B64U 20/87: Mounting of imaging devices, e.g. mounting of gimbals
- B64U 2101/30: for imaging, photography or videography
- B64U 2101/32: for cartography or topography
- B64U 2201/10: autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS]
- B64U 2201/20: Remote controls
- G01S 19/13: Receivers
- G06T 2207/10016: Video; Image sequence
- G06T 2207/10032: Satellite or aerial image; Remote sensing
- G06T 2207/20012: Locally adaptive
- G06T 2207/30232: Surveillance
- G06T 2207/30244: Camera pose
- G06V 2201/10: Recognition assisted with metadata
- H04W 4/021: Services related to particular areas, e.g. point of interest [POI] services, venue services or geofences
- G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93})
- B64C: AEROPLANES; HELICOPTERS
- B64D: EQUIPMENT FOR FITTING IN OR TO AIRCRAFT; FLIGHT SUITS; PARACHUTES; ARRANGEMENT OR MOUNTING OF POWER PLANTS OR PROPULSION TRANSMISSIONS IN AIRCRAFT
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04B: TRANSMISSION
- B64U: UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72)

## Abstract

Disclosed in this specification are methods, systems and apparatus, including computer programs encoded on non-transitory computer storage media for unmanned aerial vehicle (UAV) flight operation and privacy controls. Based on geofence types, and UAV distance from a geofence, sensors and other devices connected to a UAV are conditionally operational. Image data collected during a UAV flight may be obfuscated by the UAV while in flight, or via a post-flight process using log data generated by the UAV.

## Claims

### Claim 1

A method, comprising:
determining, in reference to a geospatial coordinate system onto which an image captured using a camera of an unmanned aerial vehicle (UAV) is projected, that a portion of the image depicts an object located outside of a geofence; and
responsive to the determining, obfuscating the portion of the image.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
projecting the image onto a geospatial coordinate system; and
overlaying the projection of the image on the geofence.

### Claim 3 (depends on 2)

The method of claim 2, wherein determining that the portion of the image depicts the object located outside of the geofence comprises:
determining an area of the geospatial coordinate system that falls outside of the geofence but is inside of the projection of the image, wherein a location of the object outside of the geofence corresponds to the area of the geospatial coordinate system; and
projecting the area of the geospatial coordinate system into a pixel coordinate system of the image, wherein the portion of the image corresponds to a location of the pixel coordinate system to which the area of the geospatial coordinate system is projected.

### Claim 4 (depends on 1)

The method of claim 1, wherein a location at which the image is captured is determined based on a flight path identified for the UAV.

### Claim 5 (depends on 4)

The method of claim 4, wherein camera parameters used by the camera of the UAV to capture the image are determined during a pre-flight simulation of the flight path.

### Claim 6 (depends on 1)

The method of claim 1, wherein the portion of the image is obfuscated while the UAV is in-flight.

### Claim 7 (depends on 6)

The method of claim 6, wherein the portion of the image is obfuscated using one or more camera parameters of the camera of the UAV.

### Claim 8 (depends on 1)

The method of claim 1, wherein the portion of the image is obfuscated via a post-flight process using log data generated by the UAV.

### Claim 9 (depends on 8)

The method of claim 8, wherein the log data includes one or more of an identification of a geospatial location at which the image was captured, a time at which the image was captured, a pose or position of the camera when the image was captured, or information indicative of or usable to determine a field of view of the camera relative to the geofence.

### Claim 10 (depends on 1)

The method of claim 1, further comprising:
delete or destroy the image responsive to a determination that a size of the portion of the image exceeds a threshold value.

### Claim 11

A system, comprising:
one or more processors of an unmanned aerial vehicle (UAV) flight processing system, the one or more processors operable to:
determine, in reference to a geospatial coordinate system onto which an image captured using a camera of the UAV is projected, that a portion of the image depicts an object located outside of a geofence; and
responsive to the determination, obfuscate the portion of the image.

### Claim 12 (depends on 11)

The system of claim 11, wherein the one or more processors are further operable to:
project the image onto a geospatial coordinate system; and
overlay the projection of the image on the geofence.

### Claim 13 (depends on 12)

The system of claim 12, wherein, to determine that the portion of the image depicts the object located outside of the geofence, the one or more processors are operable to:
determine an area of the geospatial coordinate system that falls outside of the geofence but is inside of the projection of the image, wherein a location of the object outside of the geofence corresponds to the area of the geospatial coordinate system; and
project the area of the geospatial coordinate system into a pixel coordinate system of the image, wherein the portion of the image corresponds to a location of the pixel coordinate system to which the area of the geospatial coordinate system is projected.

### Claim 14 (depends on 11)

The system of claim 11, wherein the portion of the image is obfuscated via a post-flight process using log data generated by the UAV.

### Claim 15 (depends on 11)

The system of claim 11, wherein the one or more processors are further operable to:
delete or destroy the image responsive to a determination that a size of the portion of the image exceeds a threshold value.

### Claim 16

A system, comprising:
one or more processors of an unmanned aerial vehicle (UAV) ground control system, the one or more processors operable to:
determine, in reference to a geospatial coordinate system onto which an image captured using a camera of the UAV is projected, that a portion of the image depicts an object located outside of a geofence; and
responsive to the determination, obfuscate the portion of the image.

### Claim 17 (depends on 16)

The system of claim 16, wherein the one or more processors are further operable to:
project the image onto a geospatial coordinate system; and
overlay the projection of the image on the geofence.

### Claim 18 (depends on 17)

The system of claim 17, wherein, to determine that the portion of the image depicts the object located outside of the geofence, the one or more processors are operable to:
determine an area of the geospatial coordinate system that falls outside of the geofence but is inside of the projection of the image, wherein a location of the object outside of the geofence corresponds to the area of the geospatial coordinate system; and
project the area of the geospatial coordinate system into a pixel coordinate system of the image, wherein the portion of the image corresponds to a location of the pixel coordinate system to which the area of the geospatial coordinate system is projected.
wherein the portion of the image is obfuscated while the UAV is in-flight.

### Claim 19 (depends on 16)

The system of claim 16, wherein a location at which the image is captured is determined based on a flight path identified for the UAV.

### Claim 20 (depends on 19)

The system of claim 19, wherein camera parameters used by the camera of the UAV to capture the image are determined during a pre-flight simulation of the flight path.
