# US 11,906,983 B2 · System and method for tracking targets

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,906,983 B2
- Document ID: US11906983B2
- Status: Granted
- Kind: B2
- Application number: 17/503,555
- Filed: 2021-10-18
- Published: 2022-02-03
- Granted: 2024-02-20
- Earliest claimed: 2016-08-06
- Assignees: Sz Dji Technology Co., Ltd.
- Applicants: Sz Dji Technology Co., Ltd.
- Inventors: Peiliang Li; Cong Zhao; Xuyang Feng; Jie Qian; Qifeng Wu
- Web page: https://patentgrep.com/patent/US11906983B2

## CPC

- G05D 1/12: Target-seeking control
- G05D 1/0094: {involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target}
- G05D 1/683: Intercepting moving targets (docking at a base station G05D1/661)
- G05D 1/689: Pointing payloads towards fixed or moving targets (positioning towed, pushed or suspended implements G05D1/672)
- B64U 10/13: Flying platforms
- B64U 20/87: Mounting of imaging devices, e.g. mounting of gimbals
- B64U 2101/31: for surveillance
- B64U 2201/10: autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS]
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- B64U: UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR

## Abstract

A method of tracking a movement of a target by an unmanned aerial vehicle (UAV), the method includes receiving, from a mobile terminal collocating with the target, absolute location coordinate data of the target, acquiring, by a camera of the UAV, image data of the target, determining, based on the absolute location coordinate data and the image data of the target, 3D location coordinate data of the target with respect to the UAV, and controlling, based on the 3D location coordinate data of the target with respect to the UAV, at least one of a direction or a speed of the movement of the UAV, to maintain a constant relative 3D position of the UAV with respect to the target.

## Claims

### Claim 1

A method of tracking a movement of a target by a movable object, the method comprising:
receiving, from a mobile terminal collocating with the target, absolute location coordinate data of the target;
acquiring, by a camera of the movable object, image data of the target;
determining a physical dimension associated with the target, including, in response to a change in a yaw angle of the camera not exceeding a threshold, averaging a current value of the physical dimension and one or more historical values of the physical dimension to determine the physical dimension;
determining, based on the absolute location coordinate data, the physical dimension associated with the target, and the image data of the target, 3D location coordinate data of the target with respect to the movable object; and
controlling, based on the 3D location coordinate data of the target with respect to the movable object, at least one of a direction or a speed of the movement of the movable object, to maintain a constant relative 3D position of the movable object with respect to the target.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the mobile terminal includes a global positioning system (GPS) receiver configured to obtain GPS data.

### Claim 3 (depends on 2)

The method according to claim 2, wherein the mobile terminal further includes an inertia measurement unit (IMU) configured to obtain motion data.

### Claim 4 (depends on 3)

The method according to claim 3, wherein the absolute location coordinate data of the target is determined based on the GPS data and the motion data.

### Claim 5 (depends on 1)

The method according to claim 1, wherein the absolute location coordinate data of the target is a two-dimensional (2D) location coordinate data of the target.

### Claim 6 (depends on 5)

The method according to claim 5, wherein determining 3D location coordinate data of the target with respect to the movable object comprises:
receiving absolute three-dimensional (3D) location coordinate data of the movable object;
determining, based on the 2D absolute location coordinate data of the target, the absolute 3D location coordinate data of the movable object, the physical dimension associated with the target, and the image data of the target, 3D location coordinate data of the target with respect to the movable object.

### Claim 7 (depends on 6)

The method according to claim 6, wherein determining 3D location coordinate data of the target with respect to the movable object comprises:
determining, based on the absolute 3D location coordinate data of the movable object and the absolute 2D location coordinate data of the target, a horizontal distance between the target and the movable object;
determining, from the image data, 2D pixel coordinate data associated with a plurality of features of the target;
determining, based on the 2D pixel coordinate data and the horizontal distance, the physical dimension associated with the target;
determining, based on the physical dimension associated with the target and at least one of the 2D pixel coordinate data or the horizontal distance, 3D location coordinate data of the target with respect to the movable object.

### Claim 8 (depends on 1)

The method according to claim 1, further comprising:
in response to determining that visual tracking is lost, determining estimated 3D location coordinate data of the target with respect to the movable object, based on historical absolute location coordinate data and image data;
controlling, based on the estimated 3D location coordinate data of the target with respect to the movable object, at least one of the direction or speed of the movement of the movable object, to find the target.

### Claim 9 (depends on 8)

The method according to claim 8, further comprising:
determining, based on extracted pixel data associated with the target, whether the visual tracking of the target is lost, the pixel data associated with the target being included in the image data.

### Claim 10 (depends on 9)

The method according to claim 9, further comprising:
in response to detecting that the pixel data for at least one of a plurality of features associated with the target is missing and physical dimensional of the target not being determined, determining that the visual tracking is lost.

### Claim 11

A tracking system configured to track a movement of a target, the tracking system comprising:
a memory storing executable instructions;
a camera configured to acquire image data of the target; and
at least one processor configured to execute the stored instructions to:
receive, from a mobile terminal collocating with the target, absolute location coordinate data of the target;
acquire, by a camera of a movable object, image data of the target;
determine a physical dimension associated with the target, including, in response to a change in a yaw angle of the camera not exceeding a threshold, averaging a current value of the physical dimension and one or more historical values of the physical dimension to determine the physical dimension;
determine, based on the absolute location coordinate data, the physical dimension associated with the target, and the image data of the target, 3D location coordinate data of the target with respect to the movable object; and
control, based on the 3D location coordinate data of the target with respect to the movable object, at least one of a direction or a speed of the movement of the movable object, to maintain a constant relative 3D position of the movable object with respect to the target.

### Claim 12 (depends on 11)

The tracking system according to claim 11, wherein the mobile terminal includes a global positioning system (GPS) receiver configured to obtain GPS data.

### Claim 13 (depends on 12)

The tracking system according to claim 12, wherein the mobile terminal further includes an inertia measurement unit (IMU) configured to obtain motion data.

### Claim 14 (depends on 13)

The tracking system according to claim 13, wherein the absolute location coordinate data of the target is determined based on the GPS data and the motion data.

### Claim 15 (depends on 11)

The tracking system according to claim 11, wherein the absolute location coordinate data of the target is a two-dimensional (2D) location coordinate data of the target.

### Claim 16 (depends on 15)

The tracking system according to claim 15, wherein the at least one processor is further configured to execute the stored instructions to:
receive absolute three-dimensional (3D) location coordinate data of the movable object;
determine, based on the 2D absolute location coordinate data of the target, the 3D location coordinate data of the movable object, the physical dimension associated with the target, and the image data of the target, 3D location coordinate data of the target with respect to the movable object.

### Claim 17 (depends on 16)

The tracking system according to claim 16, wherein the at least one processor is further configured to execute the stored instructions to:
determine, based on the absolute 3D location coordinate data of the movable object and the absolute 2D location coordinate data of the target, a horizontal distance between the target and the movable object;
determine, from the image data, 2D pixel coordinate data associated with a plurality of features of the target;
determine, based on the 2D pixel coordinate data and the horizontal distance, the physical dimension associated with the target;
determine, based on the physical dimension associated with the target and at least one of the 2D pixel coordinate data or the horizontal distance, 3D location coordinate data of the target with respect to the movable object.

### Claim 18 (depends on 11)

The tracking system according to claim 11, wherein the at least one processor is further configured to execute the stored instructions to:
in response to determining that visual tracking is lost, determine estimated 3D location coordinate data of the target with respect to the movable object, based on historical absolute location coordinate data and image data;
control, based on the estimated 3D location coordinate data of the target with respect to the movable object, at least one of the direction or speed of the movement of the movable object, to find the target.

### Claim 19 (depends on 18)

The tracking system according to claim 18, wherein the at least one processor is further configured to execute the stored instructions to:
determine, based on extracted pixel data associated with the target, whether the visual tracking of the target is lost, the pixel data associated with the target being included in the image data.

### Claim 20 (depends on 19)

The tracking system according to claim 19, wherein the at least one processor is further configured to execute the stored instructions to:
in response to detecting that the pixel data for at least one of a plurality of features associated with the target is missing and physical dimensional of the target not being determined, determine that the visual tracking is lost.