# US 12,007,794 B2 · Method and apparatus for tracking moving target and unmanned aerial vehicle

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,007,794 B2
- Document ID: US12007794B2
- Status: Granted
- Kind: B2
- Application number: 17/643,696
- Filed: 2021-12-10
- Published: 2022-05-05
- Granted: 2024-06-11
- Earliest claimed: 2020-06-12
- Assignees: Autel Robotics Co., Ltd.
- Applicants: Autel Robotics Co., Ltd.
- Inventors: Xin Huang; Yuanyuan Du
- Web page: https://patentgrep.com/patent/US12007794B2

## CPC

- G05D 1/0094: {involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target}
- G05D 1/686: Maintaining a relative position with respect to moving targets, e.g. following animals or humans (for pointing payloads towards targets G05D1/689; involving controlling the position or course of two or more vehicles G05D1/69)
- B64C 39/024: {of the remote controlled vehicle type, i.e. RPV}
- B64D 47/08: Arrangements of cameras
- B64U 20/87: Mounting of imaging devices, e.g. mounting of gimbals
- G05D 1/101: {specially adapted for aircraft}
- G05D 1/224: Output arrangements on the remote controller, e.g. displays, haptics or speakers
- G05D 1/689: Pointing payloads towards fixed or moving targets (positioning towed, pushed or suspended implements G05D1/672)
- G06T 7/246: using feature-based methods, e.g. the tracking of corners or segments
- G06T 7/251: {involving models}
- G06T 7/75: {involving models}
- G06V 20/17: taken from planes or by drones
- H04N 23/60: Control of cameras or camera modules
- H04N 23/675: {comprising setting of focusing regions}
- H04N 23/695: Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
- B64U 10/14: with four distinct rotor axes, e.g. quadcopters
- B64U 2101/30: for imaging, photography or videography
- B64U 2201/10: autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS]
- B64U 2201/20: Remote controls
- G06T 2207/10032: Satellite or aerial image; Remote sensing
- G06V 2201/07: Target detection
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- B64C: AEROPLANES; HELICOPTERS
- B64D: EQUIPMENT FOR FITTING IN OR TO AIRCRAFT; FLIGHT SUITS; PARACHUTES; ARRANGEMENT OR MOUNTING OF POWER PLANTS OR PROPULSION TRANSMISSIONS IN AIRCRAFT
- B64U: UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Embodiments of the disclosure disclose a method and an apparatus for tracking a moving target and an unmanned aerial vehicle. The method includes: identifying and locking a moving target; and adjusting a moving state of the unmanned aerial vehicle and/or parameters of the photographing device according to a moving state of the moving target, so that the moving target is always located on a display screen of a control terminal. A method and an apparatus for tracking a moving target and an unmanned aerial vehicle provided in the disclosure can adjust the moving state of the unmanned aerial vehicle and/or the parameters of the photographing device in real time according to the moving state of the moving target and can ensure that the moving target in a preset size is always located on the display screen of the control terminal.

## Claims

### Claim 1

A method for tracking a moving target, performed by an unmanned aerial vehicle, wherein the unmanned aerial vehicle comprises a photographing device and the photographing device comprises a gimbal and a camera mounted on the gimbal, wherein the method comprises:
identifying and locking a moving target; and
adjusting a moving state of the unmanned aerial vehicle and parameters of the photographing device according to a moving state of the moving target, so that the moving target is always located on a display screen of a control terminal, wherein the control terminal is communicably connected to the unmanned aerial vehicle;
wherein the moving state of the moving target comprises a moving direction of the moving target, the parameters of the photographing device comprise an angle of the gimbal and then;
the adjusting a moving state of the unmanned aerial vehicle and parameters of the photographing device according to a moving state of the moving target, so that the moving target is always located on a display screen of a control terminal comprises;
determining whether the moving direction of the moving target is consistent with a moving direction of the unmanned aerial vehicle; and
adjusting the angle of the gimbal according to the moving direction of the moving target, when the moving direction of the moving target is inconsistent with the moving direction of the unmanned aerial vehicle, so that the moving target is always located at a preset position on the display screen of the control terminal;
determining whether en image size of the moving target on the display screen of the control terminal is smaller than a preset size; and
adjusting the moving state of the unmanned aerial vehicle and the parameters of the photographing device, when the image size of the moving target on the display screen of the control terminal is smaller than the preset size, so that the moving target in the preset size is always located at the preset position on the display screen of the control terminal;
determining whether a speed change rate of the moving target in the moving direction of the unmanned aerial vehicle is greater than a first threshold; and
adjusting a speed of the unmanned aerial vehicle, when the speed change rate of the moving target in the moving direction of the unmanned aerial vehicle is greater than the first threshold, so that the moving target in the preset size is always located at the preset position on the display screen.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the identifying and locking a moving target comprises:
identifying and locking the moving target through machine vision.

### Claim 3 (depends on 1)

The method according to claim 1, wherein the identifying and locking a moving target comprises:
acquiring a plurality of images of the moving target through the photographing device;
extracting features of the moving target based on the plurality of images of the moving target, wherein the features of the moving target comprise at least one of a texture feature, a shape feature, or a spatial feature;
establishing a feature classification model, wherein the feature classification model is configured to represent a category of the moving target; and
performing feature matching on the moving target based on the feature classification model, to identify and lock the moving target.

### Claim 4 (depends on 3)

The method according to claim 3, wherein the method further comprises:
performing preprocessing on the plurality of images of the moving target.

### Claim 5 (depends on 4)

The method according to claim 4, wherein the performing preprocessing on the plurality of images of the moving target comprises:
performing preprocessing on performance features of each of the plurality of images of the moving target, so that the performance features of each of the images are consistent, wherein the performance features comprise at least one of the following:
a color of the image, brightness of the image, or a size of the image.

### Claim 6 (depends on 1)

The method according to claim 1, wherein when the moving direction of the moving target is inconsistent with the moving direction of the unmanned aerial vehicle, the adjusting the angle of the gimbal according to the moving direction of the moving target, so that the moving target is always located at a preset position on the display screen of the control terminal comprises:
adjusting a yaw angle of the gimbal when the moving direction of the moving target is leftward and rightward relative to the camera, so that the moving target is always located at the preset position on the display screen of the control terminal.

### Claim 7 (depends on 1)

The method according to claim 1, wherein when the moving direction of the moving target is inconsistent with the moving direction of the unmanned aerial vehicle, the adjusting the angle of the gimbal according to the moving direction of the moving target, so that the moving target is always located at a preset position on the display screen of the control terminal comprises:
adjusting a pitch angle of the gimbal when the moving direction of the moving target is forward and backward relative to the camera, so that the moving target is always located at the preset position on the display screen of the control terminal.

### Claim 8 (depends on 1)

The method according to claim 1, wherein when the image size of the moving target on the display screen of the control terminal is smaller than the preset size, the adjusting the moving state of the unmanned aerial vehicle and the parameters of the photographing device, so that the moving target in the preset size is always located at the preset position on the display screen of the control terminal comprises:
determining whether a difference between a speed of the moving target and a speed of the unmanned aerial vehicle is greater than a second threshold; and
adjusting a focal length of the camera, when the speed change rate of the moving target in the moving direction of the unmanned aerial vehicle is greater than the second threshold, so that the moving target in the preset size is always located at the preset position on the display screen.

### Claim 9 (depends on 1)

The method according to claim 1, wherein the preset position is a central position of the display screen of the control terminal.

### Claim 10 (depends on 1)

The method according to claim 1, wherein the method further comprises:
determining whether the moving target is lost; and
searching for the moving target and re-identifying and locking the moving target when the moving target is lost.

### Claim 11

An unmanned aerial vehicle, comprising:
a body;
arms, connected to the body;
a power apparatus, disposed on the arm and configured to provide power for the unmanned aerial vehicle to fly;
a photographing device, disposed on the body, wherein the photographing device comprises a gimbal and a camera mounted on the gimbal;
a vision chip, wherein the vision chip is disposed in the body and is configured to identify and lock a moving target; and
a flight controller, wherein the flight controller is disposed on the body and is communicably connected to the vision chip and the flight controller is configured to:
adjust a moving state of the unmanned aerial vehicle and parameters of the photographing device according to a moving state of the moving target, so that the moving target is always located on a display screen of a control terminal, wherein the control terminal is communicably connected to the unmanned aerial vehicle;
wherein the moving state of the moving target comprises a moving direction of the moving target, the parameters of the photographing device comprise an angle of the gimbal and then;
the flight, controller is configured to:
determine whether the moving direction of the moving target is consistent with a moving direction of the unmanned aerial vehicle; and
adjust the angle of the gimbal according to the moving direction of the moving target, when the moving direction of the moving target is inconsistent with the moving direction of the unmanned aerial vehicle, so that the moving target is always located at a preset position on the display screen of the control terminal;
determine whether an image size, of the moving, target on the display screen of the control terminal is smaller than a preset size; and
adjust the moving state of the unmanned aerial vehicle and the parameters of the photographing device, when the image size of the moving target on the display screen of the control terminal is smaller than the preset size, so that the moving target in the preset size is always located et the preset position on the display screen of the control terminal;
determine whether a speed change rate of the moving target in the moving direction of the unmanned aerial vehicle is greater than a first threshold; and
adjust a speed of the unmanned aerial vehicle, when the speed change rate of the moving target in the moving direction of the unmanned aerial vehicle is greater than the first threshold, so that the moving target in the preset size is always located at the preset position on the display screen.

### Claim 12 (depends on 11)

The unmanned aerial vehicle according to claim 11, wherein the vision chip identifies and locks the moving target through machine vision.

### Claim 13 (depends on 11)

The unmanned aerial vehicle according to claim 11, wherein the vision chip is further configured to:
acquire a plurality of images of the moving target through the photographing device;
extract features of the moving target based on the plurality of images of the moving target, wherein the features of the moving target comprise at least one of a texture feature, a shape feature, or a spatial feature;
establish a feature classification model, wherein the feature classification model is configured to represent a category of the moving target; and
perform feature matching on the moving target based on the feature classification model, to identify and lock the moving target.

### Claim 14 (depends on 13)

The unmanned aerial vehicle according to claim 13, wherein the vision chip is further configured to;
perform preprocessing on the plurality of images of the moving target.

### Claim 15 (depends on 14)

The unmanned aerial vehicle according to claim 14, wherein the vision chip is further configured to:
performing preprocessing on performance features of each of the plurality of images of the moving target, so that the performance features of each of the images are consistent, wherein the performance features comprise at least one of the following:
a color of the image, brightness of the image, or a size of the image.

### Claim 16 (depends on 11)

The unmanned aerial vehicle according to claim 11, wherein the flight controller is further configured to:
adjust a yaw angle of the gimbal when the moving direction of the moving target is leftward and rightward relative to the camera, so that the moving target is always located at the preset position on the display screen of the control terminal.

### Claim 17 (depends on 11)

The unmanned aerial vehicle according to claim 11, wherein the flight controller is further configured to:
adjust a pitch angle of the gimbal when the moving direction of the moving target is forward and backward relative to the camera, so that the moving target is always located at the preset position on the display screen of the control terminal.

### Claim 18 (depends on 11)

The unmanned aerial vehicle according to claim 11, wherein the flight controller is further configured to:
determine whether a difference between a speed of the moving target and a speed of the unmanned aerial vehicle is greater than a second threshold; and
adjust a focal length of the camera, when the speed change rate of the moving target in the moving direction of the unmanned aerial vehicle is greater than the second threshold, so that the moving target in the preset size is always located at the preset position on the display screen.

### Claim 19 (depends on 11)

The unmanned aerial vehicle according to claim 11, wherein the preset position is a central position of the display screen of the control terminal.

### Claim 20 (depends on 11)

The unmanned aerial vehicle according to claim 11, wherein the vision chip is further configured to:
determine whether the moving target is lost;
search for the moving target and re-identify and lock the moving target when the moving target is lost.