# US 2018/0022472 A1 · Autonomous system for taking moving images from a drone, with target tracking and improved target location

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2018/0022472 A1
- Document ID: US20180022472A1
- Status: Published application
- Kind: A1
- Application number: 15/657,145
- Filed: 2017-07-22
- Published: 2018-01-25
- Earliest claimed: 2016-07-22
- Assignees: Parrot Drones
- Applicants: Parrot Drones
- Inventors: Laure CHEVALLEY; Edouard Leurent
- Web page: https://patentgrep.com/patent/US20180022472A1

## CPC

- H04N 17/002: {for television cameras}
- G01S 17/86: Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
- G05D 1/0094: {involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target}
- H04N 23/54: Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
- H04N 23/58: Means for changing the camera field of view without moving the camera body, e.g. nutating or panning of optics or image sensors
- H04N 23/698: for achieving an enlarged field of view, e.g. panoramic image capture
- B64U 2101/30: for imaging, photography or videography
- B64U 2201/10: autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS]
- B64U 2201/104: using satellite radio beacon positioning systems, e.g. GPS
- B64U 2201/20: Remote controls
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- B64U: UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR

## Abstract

The displacements of the drone are defined by piloting commands to take moving images of a target carrying the ground station. The system comprises means for adjusting the sight angle of the camera during the displacements of the drone and of the target, so that the images are centred to the target, and means for generating flying instructions so that the distance between drone and target fulfils determined rules, these means being based on a determination of the GPS geographical position of the target with respect to the GPS geographical position of the drone, and of the angular position of the target with respect to a main axis of the drone. These means are also based on the analysis of a non-geographical signal produced by the target and received by the drone. The system allows freeing from the uncertainty of the GPS systems equipping this type of device.

## Claims

### Claim 1

A system for taking moving images, comprising:
a drone provided with a camera, and a ground station communicating with the drone through a wireless link,
the displacements of the drone being defined by flying instructions applied to a propulsion unit or a set of propulsion units of the drone, the drone being adapted to fly autonomously to take moving images of a target moving with the ground station,
a processor and memory; and,
a drone management module comprising program code executing by the processor to perform:
adjusting the sight angle of the camera during the displacements of the drone and of the target so that the images taken by the camera are centred to the target; and
generating said flying instructions to control the displacement of the drone towards the target and so that the distance and the angular position between the drone and the target fulfil determined rules,
the adjusting the sight angle and for generating said flying instructions being based:
on a determination of the GPS geographical position of the target with respect to the GPS geographical position of the drone, and of the angular position of the target with respect to a main axis linked to the drone, and
on the analysis of a signal produced by analysis of the images successively delivered by the camera of the drone and/or produced by the target and received by the drone,
a merging software engine adapted to merge, in order to obtain refined information about the geographical position of the target:
information about the geographical position of the target, also comprising an associated value of uncertainty of the said geographical position, and
information about the angular position of the target with respect to a main axis linked to the drone, also comprising an associated value of uncertainty of said angular position,
the merging software engine comprising a hybrid-coordinates extended Kalman filter, comprising:
a prediction unit sensitive to a model of prediction of the target movement and operating based on:
adjusted estimates of the target position and speed obtained based on said target geographical position information and
adjusted estimates of the target position and speed obtained based on said angular position information, as well as respective uncertainties of said estimates;
a first adjustment unit receiving as an input the target geographical position and speed and estimates of the target position and speed coming from the prediction unit, as well as their respective uncertainties, and delivering as an output adjusted values of the target position and speed, as well as their respective uncertainties; and
a second adjustment unit receiving as an input the target angular position and speed and estimates of the target angular position and speed coming from the prediction unit, as well as their respective uncertainties, and delivering as an output adjusted values of the target position and speed, as well as their respective uncertainties.

### Claim 2 (depends on 1)

The system according to claim 1, wherein the prediction model is a constant target speed model.

### Claim 3 (depends on 1)

The system according to claim 1, wherein the merging software engine further comprises a conversion unit converting the target position and speed information, generated in Cartesian coordinates, into corresponding information converted into angular coordinates, and vice versa, and in that said second adjustment unit operates in angular coordinates.

### Claim 4 (depends on 1)

The system according to claim 1, wherein the merging software engine in performing merging takes into account at least one additional piece of information among:
information about the measured altitude of the target;
information about the measured acceleration of the target;
information about the measured distance between the drone and the target; and
additional information about a scale or a scale change in the images taken by the camera.

### Claim 5 (depends on 1)

The system according to claim 1, wherein the rate of analysis of the signal produced by analysis of the images successively delivered by the camera of the drone, and/or produced by the target and received by the drone, is higher than the rate at which the GPS geographical positions are determined.

### Claim 6 (depends on 1)

The system according to claim 1, wherein the signal produced by analysis of the images successively delivered by the camera of the drone and/or produced by the target and received by the drone, is a visual signal of position of the target in the images generated by the camera of the drone, and in that it includes means for locating and following in the successive images the location of said signal.

### Claim 7 (depends on 6)

The system according to claim 6, wherein the camera is a wide-angle fixed camera of the fisheye type and in that the means for adjusting the sight angle of the camera are adapted to frame and process the wide-angle images taken by the camera.

### Claim 8 (depends on 6)

The system according to claim 6, wherein the camera is a camera mounted on a three-axis articulated support of the gimbal type with Cardan suspension, provided with servomotors piloted by said means for adjusting the sight angle of the camera.

### Claim 9 (depends on 1)

The system according to claim 1, wherein the signal produced by analysis of the images successively delivered by the camera of the drone and/or produced by the target and received by the drone is a signal of the lidar type reflected by the target from an emission by the drone.
