# US 12,007,763 B2 · Magic wand interface and other user interaction paradigms for a flying digital assistant

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,007,763 B2
- Document ID: US12007763B2
- Status: Granted
- Kind: B2
- Application number: 18/162,122
- Filed: 2023-01-31
- Published: 2023-09-07
- Granted: 2024-06-11
- Earliest claimed: 2014-06-19
- Assignees: Skydio, Inc.
- Applicants: Skydio, Inc.
- Inventors: Abraham Bachrach; Adam Bry; Matthew Donahoe
- Web page: https://patentgrep.com/patent/US12007763B2

## CPC

- G05D 1/0016: {characterised by the operator's input device (input arrangements for computing systems in general G06F3/00)}
- B64C 39/024: {of the remote controlled vehicle type, i.e. RPV}
- B64D 47/08: Arrangements of cameras
- G01C 21/1656: {with passive imaging devices, e.g. cameras}
- G01C 21/20: Instruments for performing navigational calculations (G01C21/24, G01C21/26 take precedence)
- G01C 23/00: Combined instruments indicating more than one navigational value, e.g. for aircraft; Combined measuring devices for measuring two or more variables of movement, e.g. distance, speed or acceleration
- G05D 1/0027: {involving a plurality of vehicles, e.g. fleet or convoy travelling (fleet control of land vehicles from a control room G05D1/0297; traffic control systems for road vehicles G08G1/00; for marine craft G08G3/00; for aircraft G08G5/00)}
- G05D 1/0044: {by providing the operator with a computer generated representation of the environment of the vehicle, e.g. virtual reality, maps (maps used for automatic navigation G05D1/0274; flight directors G01C23/005)}
- G05D 1/0094: {involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target}
- G05D 1/223: Command input arrangements on the remote controller, e.g. joysticks or touch screens
- G05D 1/225: operated by off-board computers
- G05D 1/689: Pointing payloads towards fixed or moving targets (positioning towed, pushed or suspended implements G05D1/672)
- G05D 1/692: involving a plurality of disparate vehicles
- B64U 10/10: Rotorcrafts
- B64U 10/13: Flying platforms
- B64U 30/20: Rotors; Rotor supports
- B64U 2101/30: for imaging, photography or videography
- B64U 2201/00: UAVs characterised by their flight controls
- B64U 2201/10: autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS]
- B64U 2201/104: using satellite radio beacon positioning systems, e.g. GPS
- B64U 2201/20: Remote controls
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- B64C: AEROPLANES; HELICOPTERS
- B64D: EQUIPMENT FOR FITTING IN OR TO AIRCRAFT; FLIGHT SUITS; PARACHUTES; ARRANGEMENT OR MOUNTING OF POWER PLANTS OR PROPULSION TRANSMISSIONS IN AIRCRAFT
- G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S)
- B64U: UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR

## Abstract

Methods and systems are described for new paradigms for user interaction with an unmanned aerial vehicle (referred to as a flying digital assistant or FDA) using a portable multifunction device (PMD) such as smart phone. In some embodiments, a magic wand user interaction paradigm is described for intuitive control of an FDA using a PMD. In other embodiments, methods for scripting a shot are described.

## Claims

### Claim 1

A method for controlling an unmanned aerial vehicle (UAV), the method comprising:
defining, by a processing system, a virtual line extending from a mobile device to the UAV; and
causing the UAV to autonomously maneuver towards a point along the virtual line.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
receiving an input indicating an intent to control the UAV.

### Claim 3 (depends on 2)

The method of claim 2, further comprising:
responsive to the input, causing the UAV to maintain its current distance relative to a reference point.

### Claim 4 (depends on 3)

The method of claim 3, further comprising:
responsive to the input, causing the UAV to maintain its current position and/or orientation relative to the reference point.

### Claim 5 (depends on 4)

The method of claim 4, wherein the reference point comprises a location of the mobile device in the physical environment.

### Claim 6 (depends on 2)

The method of claim 2, further comprising:
responsive to the input, causing the UAV to autonomously maneuver to a pre-defined position and/or orientation relative to a reference point.

### Claim 7 (depends on 6)

The method of claim 6, further comprising:
tracking, by the processing system, a location of the mobile device in the physical environment;
wherein the reference point comprises the location of the mobile device.

### Claim 8 (depends on 6)

The method of claim 6, wherein the reference point is at or associated with a physical object in physical space.

### Claim 9 (depends on 1)

The method of claim 1, further comprising:
receiving an input indicating an intent to adjust a distance between the UAV and a reference point while maintaining a same angle of elevation.

### Claim 10 (depends on 9)

The method of claim 9, wherein the input is received via a touch screen of the mobile device.

### Claim 11 (depends on 10)

The method of claim 10, wherein the adjustment to the distance between the UAV and the reference point can be manipulated via a slider bar on the touch screen.

### Claim 12

A system for controlling an unmanned aerial vehicle (UAV), the system comprising:
a display;
one or more processors; and
one or more memory units having instructions stored thereon which, when executed by the one or more processors, cause the system to:
define a virtual line extending from the system to the UAV; and
generate control commands configured to cause the UAV to autonomously maneuver towards a point along the virtual line; and
transmit the control commands to the UAV.

### Claim 13 (depends on 12)

The system of claim 12, further comprising:
an input mechanism configured to receive an input indicating an intent to control the UAV.

### Claim 14 (depends on 13)

The system of claim 13, wherein the instructions, when executed by the one or more processors, further cause the system to:
responsive to the input, generate instructions configured to cause the UAV to maintain its current distance relative to a reference point; and
transmit the instructions to the UAV.

### Claim 15 (depends on 13)

The system of claim 13, wherein the instructions, when executed by the one or more processors, further cause the system to:
responsive to the input, generate instructions configured to cause the UAV to maintain its current position and/or orientation relative to the reference point; and
transmit the instructions to the UAV.

### Claim 16 (depends on 13)

The system of claim 13, wherein the instructions, when executed by the one or more processors, further cause the system to:
responsive to the input, generate instructions configured to cause the UAV to autonomously maneuver to a pre-defined position and/or orientation relative to a reference point; and
transmit the instructions to the UAV.

### Claim 17 (depends on 13)

The system of claim 13, wherein the input mechanism is further configured to receive an input indicating an intent to adjust a distance between the UAV and a reference point while maintaining a same angle of elevation.

### Claim 18 (depends on 17)

The system of claim 17, wherein the input mechanism comprises a touch screen.

### Claim 19 (depends on 18)

The system of claim 18, wherein the adjustment to the distance between the UAV and the reference point is manipulated via a slider bar presented on the touch screen.

### Claim 20

An unmanned aerial vehicle (UAV) configured for autonomous flight through a physical environment, the UAV comprising:
a first image capture device configured to capture images of the physical environment;
a propulsion system configured to maneuver the UAV through the physical environment;
a processing system; and
an autonomous navigation system configured to:
define a virtual line extending from a mobile device to the UAV; and
cause the propulsion system to autonomously maneuver the UAV towards a point along the virtual line.