# US 2019/0378250 A1 · Image processing method, device and photographic apparatus

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0378250 A1
- Document ID: US20190378250A1
- Status: Published application
- Kind: A1
- Application number: 16/551,375
- Filed: 2019-08-26
- Published: 2019-12-12
- Earliest claimed: 2014-12-09
- Assignees: Sz Dji Technology Co., Ltd.
- Applicants: Sz Dji Technology Co., Ltd.
- Inventors: Guyue ZHOU; Ketan TANG
- Web page: https://patentgrep.com/patent/US20190378250A1

## CPC

- G06T 5/77: Retouching; Inpainting; Scratch removal
- G06T 5/80: Geometric correction
- G06T 7/13: Edge detection
- G06T 3/4007: based on interpolation, e.g. bilinear interpolation (image demosaicing G06T3/4015; edge-driven or edge-based scaling G06T3/403)
- G06T 2207/10016: Video; Image sequence
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

An image processing method includes correcting a target image based on an initial distortion coefficient to obtain a first corrected target image, performing straight-line fitting on a first border line in the first corrected target image to calculate a first distortion metric value and a correction distortion coefficient, correcting the target image based on the correction distortion coefficient to obtain a second corrected target image, removing outlier points on a second border line in the second corrected target image, performing straight-line fitting on the second border line with the outlier points removed to calculate a second distortion metric value, detecting whether a preset correction condition is satisfied based on at least one of the first distortion metric value or the second distortion metric value, and, if the preset correction condition is satisfied, applying the correction distortion coefficient to subsequent image correction to obtain better corrected images.

## Claims

### Claim 1

An image processing method comprising:
correcting a target image based on an initial distortion coefficient to obtain a first corrected target image;
performing straight-line fitting on a first border line in the first corrected target image to calculate a first distortion metric value and a correction distortion coefficient;
correcting the target image based on the correction distortion coefficient to obtain a second corrected target image;
removing outlier points on a second border line in the second corrected target image;
performing straight-line fitting on the second border line with the outlier points removed to calculate a second distortion metric value;
detecting whether a preset correction condition is satisfied based on at least one of the first distortion metric value or the second distortion metric value; and
applying, if the preset correction condition is satisfied, the correction distortion coefficient to subsequent image correction to obtain better corrected images.

### Claim 2 (depends on 1)

The method according to claim 1, further comprising, before correcting the target image:
capturing an image of an object including straight line features; and
using the captured image as the target image or adjusting a size of the captured image to obtain the target image.

### Claim 3 (depends on 2)

The method according to claim 2, wherein capturing the image of the object including straight line features includes:
capturing a plurality of images; and
analyzing the plurality of images to determine the image of the object including straight line features.

### Claim 4 (depends on 3)

The method according to claim 3, wherein analyzing the plurality of images includes analyzing the plurality of images at a same time using a same processing method.

### Claim 5 (depends on 2)

The method according to claim 2, wherein adjusting the size of the captured image includes:
magnifying, if the size of the captured image is smaller than a preset size threshold, the captured image to a target size through interpolation; or
scaling down, if the size of the captured image is greater than the preset size threshold, the captured image to the target size through down-sampling.

### Claim 6 (depends on 1)

The method according to claim 1, wherein performing straight-line fitting on the first border line to calculate the first distortion metric value and the correction distortion coefficient includes:
performing edge detection on the first corrected target image to determine the first border line in the first corrected target image;
performing straight-line fitting on the first border line based on polynomial straight-line fitting to obtain a fitted straight line; and
calculating the first distortion metric value of the first border line relative to the fitted straight line and the correction distortion coefficient corresponding to the first distortion metric value.

### Claim 7 (depends on 6)

The method according to claim 6, wherein calculating the first distortion metric value and the correction distortion coefficient includes:
determining a straight line segment in the first border line;
calculating distances from corresponding points on the straight line segment to the fitted straight line;
obtaining the first distortion metric value according to the distances; and
performing non-linear optimization on the first distortion metric value to obtain the correction distortion coefficient.

### Claim 8 (depends on 1)

The method according to claim 1, wherein:
correcting the target image according to the initial distortion coefficient includes correcting a target border line in the target image based on the initial distortion coefficient, and
correcting the target image based on the correction distortion coefficient includes correcting the target border line in the target image based on the correction distortion coefficient.

### Claim 9 (depends on 1)

The method according to claim 1, wherein performing straight-line fitting on the second border line with the outlier points removed to calculate the second distortion metric value includes:
performing edge detection on the second corrected target image to determine the second border line in the second corrected target image;
performing straight-line fitting on the second border line based on polynomial straight-line fitting to obtain a fitted straight line; and
calculating the second distortion metric value of the second border line relative to the fitted straight line.

### Claim 10 (depends on 9)

The method according to claim 9, wherein calculating the second distortion metric value includes:
determining a straight line segment in the second border line with the outlier points removed;
calculating distances from corresponding points on the straight line segment to the fitted straight line; and
obtaining the second distortion metric value according to the distances.

### Claim 11 (depends on 1)

The method according to claim 1, wherein detecting whether the preset correction condition is satisfied includes:
calculating a relative variation amount between the first distortion metric value and the second distortion metric value; and
determining whether the relative variation amount calculated is smaller than a preset variation threshold to determine whether the preset correction condition is satisfied.

### Claim 12 (depends on 1)

The method according to claim 1, wherein detecting whether the preset correction condition is satisfied includes determining whether the second distortion metric value is smaller than a preset metric threshold.

### Claim 13 (depends on 1)

The method according to claim 1, further comprising:
configuring, if the preset correction condition is not satisfied, the correction distortion coefficient as the initial distortion coefficient.

### Claim 14

A camera comprising:
a camera lens; and
an image processor configured to:
correct a target image based on an initial distortion coefficient to obtain a first corrected target image;
perform straight-line fitting on a first border line in the first corrected target image to calculate a first distortion metric value and a correction distortion coefficient;
correct the target image based on the correction distortion coefficient to obtain a second corrected target image;
remove outlier points on a second border line in the second corrected target image;
perform straight-line fitting on the second border line with the outlier points removed to calculate a second distortion metric value;
detect whether a preset correction condition is satisfied based on at least one of the first distortion metric value or the second distortion metric value; and
apply, if the preset correction condition is satisfied, the correction distortion coefficient to subsequent image correction to obtain better corrected images.

### Claim 15 (depends on 14)

The camera according to claim 14, wherein the image processor is further configured to:
capture an image of an object including straight line features through the camera lens; and
determine the captured image as the target image or adjust a size of the captured image to obtain the target image.

### Claim 16 (depends on 15)

The camera according to claim 15, wherein the image processor is further configured to:
capture a plurality of images; and
analyze the plurality of images to determine the image of the object including straight line features.

### Claim 17 (depends on 15)

The camera according to claim 15, wherein the image processor is further configured to analyze the plurality of images at a same time using a same processing method.

### Claim 18 (depends on 15)

The camera according to claim 15, wherein the image processor is further configured to:
magnify, if the size of the captured image is smaller than a preset size threshold, the captured image to a target size through interpolation; or
scale down, if the size of the captured image is greater than the preset size threshold, the captured image to the target size through down-sampling.

### Claim 19 (depends on 14)

The camera according to claim 14, wherein the image processor is further configured to:
perform edge detection on the second corrected target image to determine the second border line in the second corrected target image;
perform straight-line fitting on the second border line based on polynomial straight-line fitting to obtain a fitted straight line; and
calculate the second distortion metric value of the second border line relative to the fitted straight line.

### Claim 20 (depends on 19)

The camera according to claim 19, wherein the image processor is further configured to:
determine a straight line segment in the second border line with the outlier points removed;
calculate distances from corresponding points on the straight line segment to the fitted straight line; and
obtain the second distortion metric value according to the distances.
