# US 10,871,258 B2 · Method and system for controlling gimbal

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,871,258 B2
- Document ID: US10871258B2
- Status: Granted
- Kind: B2
- Application number: 16/265,595
- Filed: 2019-02-01
- Published: 2019-06-13
- Granted: 2020-12-22
- Earliest claimed: 2016-08-03
- Assignees: Sz Dji Osmo Technology Co., Ltd.
- Applicants: Sz Dji Osmo Technology Co., Ltd.
- Inventors: Chenglin Mao; Linchao Bao; Pan Hu; Zisheng Cao
- Web page: https://patentgrep.com/patent/US10871258B2

## CPC

- H04N 23/611: where the recognised objects include parts of the human body
- B64U 20/87: Mounting of imaging devices, e.g. mounting of gimbals
- F16M 11/123: {the axis of rotation intersecting in a single point, e.g. by using gimbals}
- G05D 1/00: Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots (drive control systems specially adapted for autonomous road vehicles B60W60/00)
- G06T 5/70: Denoising; Smoothing
- H04N 23/54: Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
- H04N 23/68: for stable pick-up of the scene, e.g. compensating for camera body vibrations
- H04N 23/695: Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
- H04N 23/80: Camera processing pipelines; Components thereof
- B64U 2101/30: for imaging, photography or videography
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- B64U: UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
- F16M: FRAMES, CASINGS OR BEDS OF ENGINES, MACHINES OR APPARATUS, NOT SPECIFIC TO ENGINES, MACHINES OR APPARATUS PROVIDED FOR ELSEWHERE; STANDS; SUPPORTS
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A method for controlling a gimbal includes obtaining an image recorded by an image sensor that is carried by the gimbal, extracting a reference object from the image, determining an offset attitude angle of the image sensor according to position information of the reference object in a preset coordinate system, and adjusting the attitude of the image sensor by the gimbal according to the offset attitude angle.

## Claims

### Claim 1

A method for controlling a gimbal comprising:
obtaining an image recorded by an image sensor that is carried by the gimbal;
performing a distortion correction on the image to generate a corrected image;
extracting a reference object from the corrected image;
determining an offset attitude angle of the image sensor according to position information of the reference object in a preset coordinate system; and
adjusting the attitude of the image sensor by the gimbal according to the offset attitude angle.

### Claim 2 (depends on 1)

The method according to claim 1, wherein extracting the reference object from the corrected image comprises:
extracting features in the corrected image;
determining characteristic values of the features; and
determining, according to the characteristic values of the features, a feature whose characteristic value satisfies a preset selection rule from the features as the reference object.

### Claim 3 (depends on 2)

The method according to claim 2, wherein the features comprise at least one of a straight line, a regular curve, or a point.

### Claim 4 (depends on 3)

The method according to claim 3, wherein when the features include straight lines, determining the characteristic values of the features comprises:
determining at least one of a grayscale of regions on two sides of one of the straight lines, a color of the regions on two sides of the one of the straight lines, a uniformity of the one of the straight lines, or an angle of the one of the straight lines.

### Claim 5 (depends on 4)

The method according to claim 4, wherein determining, according to the characteristic values of the features, the feature whose characteristic value satisfies the preset selection rule from the features as the reference object, comprises:
selecting, from the straight lines, a straight line whose characteristic value is greater than a preset threshold as the reference object.

### Claim 6 (depends on 5)

The method according to claim 5, wherein selecting the straight line whose characteristic value is greater than the preset threshold as the reference object, comprises:
identifying, using a least square method, a straight line having a smallest error when being simultaneously parallel or perpendicular to each of straight lines having characteristic values greater than the preset threshold as the reference object.

### Claim 7 (depends on 1)

The method according to claim 1, wherein before extracting the reference object from the corrected image, the method further comprises:
performing a projection transformation on the corrected image according to an attitude angle of the image sensor when the image sensor is recording the image.

### Claim 8 (depends on 7)

The method according to claim 7, wherein:
the attitude angle comprises a pitch angle and a roll angle; and
performing the projection transformation on the corrected image according to the attitude angle of the image sensor when the image sensor is recording the image, comprises:
calculating a first rotation matrix according to the pitch angle of the image sensor when the image sensor is recording the image and a second rotation matrix according to the roll angle of the image sensor when the image sensor is recording the image;
multiplying the first rotation matrix and the second rotation matrix to obtain an overall rotation matrix; and
performing a coordinate transformation on coordinates of pixels on the image, according to the overall rotation matrix.

### Claim 9 (depends on 1)

The method according to claim 1, wherein determining the offset attitude angle of the image sensor, according to the position information of the reference object in the preset coordinate system, comprises:
calculating a rotation matrix of the reference object in the preset coordinate system with respect to a specific direction; and
decomposing the rotation matrix to obtain the offset attitude angle of the image sensor.

### Claim 10 (depends on 9)

The method according to claim 9, wherein the specific direction comprises at least one of a horizontal direction, a vertical direction, or a preset direction detected by the attitude sensor.

### Claim 11 (depends on 9)

The method according to claim 9, wherein before calculating the rotation matrix of the reference object in the preset coordinate system with respect to the specific direction, the method further comprises:
performing an inverse projection transformation on the reference object according to the attitude angle of the image sensor when the image sensor is recording the image.

### Claim 12 (depends on 9)

The method according to claim 9, wherein adjusting the attitude of the image sensor by the gimbal, according to the offset attitude angle, comprises:
controlling a power apparatus of the gimbal to adjust the attitude of the image sensor, such that the reference object in the image recorded by the adjusted image sensor coincides with the specific direction.

### Claim 13

A method for controlling an electronic image stabilization system comprising:
obtaining an image in a capture area on an image sensor;
performing a distortion correction on the image to generate a corrected image;
extracting a reference object from the corrected image;
determining an offset attitude angle of a view angle corresponding to the capture area, according to position information of the reference object in a preset coordinate system; and
adjusting a position of the capture area, according to the offset attitude angle.

### Claim 14 (depends on 13)

The method according to claim 13, wherein extracting the reference object from the corrected image comprises:
extracting features in the corrected image;
determining characteristic values of the features; and
determining, according to the characteristic values of the features, the feature whose characteristic value satisfies a preset selection rule from the features as the reference object.

### Claim 15 (depends on 13)

The method according to claim 13, wherein before extracting the reference object from the corrected image, the method further comprises:
performing a projection transformation on the corrected image, according to the attitude angle of the view angle corresponding to the capture area when the capture area is capturing the image.