# US 2023/0058428 A1 · Method and apparatus for automatic parking system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0058428 A1
- Document ID: US20230058428A1
- Status: Published application
- Kind: A1
- Application number: 17/820,212
- Filed: 2022-08-16
- Published: 2023-02-23
- Earliest claimed: 2021-08-20
- Assignees: Harman International Industries, Incorporated
- Applicants: Harman International Industries, Incorporated
- Inventors: Chuan XIAO
- Web page: https://patentgrep.com/patent/US20230058428A1

## CPC

- G06T 7/11: Region-based segmentation
- G06V 20/586: {of parking space}
- G06V 10/751: {Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching}
- G06V 10/82: using neural networks
- G06T 2207/20044: Skeletonization; Medial axis transform
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30264: Parking
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

The present disclosure provides a method and apparatus for an automatic parking system. The method includes: obtaining a parking-line segmentation image through a segmentation neural network; preprocessing the parking-line segmentation image to obtain at least one parking-line skeleton image; and calculating straight-line equations and straight-line endpoints for parking lines in the parking-line segmentation image based on the at least.

## Claims

### Claim 1

A method for an automatic parking system, comprising:
obtaining a parking-line segmentation image through a segmentation neural network;
preprocessing the parking-line segmentation image to obtain at least one parking-line skeleton image; and
calculating straight-line equations and straight-line endpoints for parking lines in the parking-line segmentation image based on the at least one parking-line skeleton image.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the preprocessing of the parking-line segmentation image to obtain at least one parking-line skeleton image comprises:
extracting pixels belonging to parking lines from the parking-line segmentation image, displaying one or more pixels belonging to parking lines with first color, and displaying one or more remaining pixels not belonging to parking lines with second color, thereby generating an extracted parking-line segmentation image; and
performing skeleton extraction on the extracted parking-line segmentation image to extract skeletons of the parking lines, thereby generating the at least one parking-line skeleton image.

### Claim 3 (depends on 1)

The method according to claim 1, wherein the preprocessing of the parking-line segmentation image to obtain at least one parking-line skeleton image comprises:
extracting pixels belonging to parking lines from the parking-line segmentation image, displaying one or more pixels belonging to parking lines with first color, and displaying one or more remaining pixels not belonging to parking lines with second color, thereby generating an extracted parking-line segmentation image;
dilating the extracted parking-line segmentation image to generate a dilated parking-line segmentation image; and
performing skeleton extraction on the dilated parking-line segmentation image to extract skeletons of the parking lines, thereby generating the at least one parking-line skeleton image.

### Claim 4 (depends on 1)

The method according to claim 1, wherein the preprocessing of the parking-line segmentation image to obtain at least one parking-line skeleton image comprises:
extracting pixels belonging to parking lines from the parking-line segmentation image, displaying one or more pixels belonging to parking lines with first color, and displaying one or more remaining pixels not belonging to parking lines with second color, thereby generating an extracted parking-line segmentation image;
dilating the extracted parking-line segmentation image to generate a dilated parking-line segmentation image;
determining at least one connection region based on the dilated parking-line segmentation image, wherein each connection region corresponds to a region in the dilated parking-line segmentation image, each connection region comprises pixels of parking lines connected to each other, and the pixels of the parking lines belonging to different connection regions are separated from each other;
dividing the dilated parking-line segmentation image according to the at least one connection region to generate sub-images respectively corresponding to different connection regions; and
performing skeleton extraction on each sub-image to generate respective parking-line skeleton images corresponding to the sub-images.

### Claim 5 (depends on 1)

The method according to claim 1, wherein the calculating of straight-line equations and straight-line endpoints for parking lines in the parking-line segmentation image based on the at least one parking-line skeleton image comprises:
performing, for each parking-line skeleton image, the following operations:
a first operation dividing the parking-line skeleton image into multiple patches;
a second operation generating multiple straight-line equations based on the multiple patches, wherein each straight-line equation is formed by fitting one or more pixels with first color in each patch;
a third operation selecting an optimal straight-line equation from the multiple straight-line equations, wherein the optimal straight-line equation represents a straight line that matches the largest quantity of pixels with first color in the parking-line skeleton image;
a fourth operation calculating corresponding endpoints of the optimal straight-line equation in the parking-line skeleton image; and
a fifth operation deleting the pixels with first color corresponding to the optimal straight-line equation from the parking-line skeleton image, and
performing the first operation, the second operation, the third operation, the fourth operation, and the fifth operation in a loop until at least one of: a number of repetitions of the first operation, the second operation, the third operation, the fourth operation, and the fifth operation reaches a straight-line quantity threshold; and a quantity of remaining pixels with first color in the parking-line skeleton image is less than a pixel quantity threshold.

### Claim 6 (depends on 5)

The method according to claim 5, wherein the generating of multiple straight-line equations based on the multiple patches comprises:
performing, for each patch, the following operations in parallel:
getting the patch;
obtaining coordinates of one or more pixels with first color in the patch; and
fitting a straight-line equation based on the obtained coordinates of the one or more pixels with first color.

### Claim 7 (depends on 6)

The method according to claim 6, further comprising:
performing, for each patch, the following operations in parallel:
determining whether a ratio of a quantity of pixels with first color matched by the straight-line equation to a quantity of all pixels with first color in the patch is greater than a ratio threshold, wherein a pixel with first color in the patch whose distance from a straight line represented by the straight-line equation is less than a distance threshold is considered to be a pixel with first color matched by the straight-line equation;
in response to the ratio being greater than the ratio threshold, transferring the straight-line equation to a coordinate system of the parking-line skeleton image to represent a straight line in the parking-line skeleton image, and taking a quantity of pixels with first color matched by the straight-line equation in the parking-line skeleton image as a value of a comparison parameter; and
setting the value of the comparison parameter to 0 in response to the ratio being equal to or less than the ratio threshold.

### Claim 8 (depends on 7)

The method according to claim 7, further comprising:
comparing multiple comparison parameter values calculated for multiple patches; and
selecting the straight-line equation corresponding to a largest comparison parameter value among the multiple comparison parameter values as the optimal straight-line equation.

### Claim 9 (depends on 2)

The method according to claim 2, further comprising:
filtering the at least one parking-line skeleton image to generate a filtered parking-line skeleton image.

### Claim 10

An automatic parking system, comprising:
one or more processors; and
a memory having executable instructions that, when executed, cause the one or more processors to:
obtain a parking-line segmentation image through a segmentation neural network;
preprocess the parking-line segmentation image to obtain at least one parking-line skeleton image; and
calculate straight-line equations and straight-line endpoints for parking lines in the parking-line segmentation image based on the at least one parking-line skeleton image.

### Claim 11 (depends on 10)

The automatic parking system according to claim 10, wherein to preprocess the parking-line segmentation image to obtain at least one parking-line skeleton image comprises:
extract pixels belonging to parking lines from the parking-line segmentation image, display one or more pixels belonging to parking lines with first color, and display one or more remaining pixels not belonging to parking lines with second color, to thereby generate an extracted parking-line segmentation image; and
perform skeleton extraction on the extracted parking-line segmentation image to extract skeletons of the parking lines, and to thereby generate the at least one parking-line skeleton image.

### Claim 12 (depends on 10)

The automatic parking system according to claim 10, wherein to preprocess the parking-line segmentation image to obtain at least one parking-line skeleton image comprises:
extract pixels belonging to parking lines from the parking-line segmentation image, display one or more pixels belonging to parking lines with first color, and display one or more remaining pixels not belonging to parking lines with second color, to thereby generate an extracted parking-line segmentation image;
dilate the extracted parking-line segmentation image to generate a dilated parking-line segmentation image; and
perform skeleton extraction on the dilated parking-line segmentation image to extract skeletons of the parking lines, and to thereby generate the at least one parking-line skeleton image.

### Claim 13 (depends on 10)

The automatic parking system according to claim 10, wherein to preprocess the parking-line segmentation image to obtain at least one parking-line skeleton image comprises:
extract pixels belonging to parking lines from the parking-line segmentation image, display one or more pixels belonging to parking lines with first color, and display one or more remaining pixels not belonging to parking lines with second color, to thereby generate an extracted parking-line segmentation image;
dilate the extracted parking-line segmentation image to generate a dilated parking-line segmentation image;
determine at least one connection region based on the dilated parking-line segmentation image, wherein each connection region corresponds to a region in the dilated parking-line segmentation image, each connection region comprises pixels of parking lines connected to each other, and the pixels of the parking lines belonging to different connection regions are separated from each other;
divide the dilated parking-line segmentation image according to the at least one connection region to generate sub-images respectively corresponding to different connection regions; and
perform skeleton extraction on each sub-image to generate respective parking-line skeleton images corresponding to the sub-images.

### Claim 14 (depends on 10)

The automatic parking system according to claim 10, wherein to calculate straight-line equations and straight-line endpoints for parking lines in the parking-line segmentation image based on the at least one parking-line skeleton image comprises:
perform, for each parking-line skeleton image, the following operations:
a first operation to divide the parking-line skeleton image into multiple patches;
a second operation to generate multiple straight-line equations based on the multiple patches, wherein each straight-line equation is formed by fitting one or more pixels with first color in each patch;
a third operation to select an optimal straight-line equation from the multiple straight-line equations, wherein the optimal straight-line equation represents a straight line that matches the largest quantity of pixels with first color in the parking-line skeleton image;
a fourth operation to calculate corresponding endpoints of the optimal straight-line equation in the parking-line skeleton image; and
a fifth operation to delete the pixels with first color corresponding to the optimal straight-line equation from the parking-line skeleton image, and
perform the first operation, the second operation, the third operation, the fourth operation, and the fifth operation in a loop until at least one of: a number of repetitions of the first operation, the second operation, the third operation, the fourth operation, and the fifth operation reaches a straight-line quantity threshold; and a quantity of remaining pixels with first color in the parking-line skeleton image is less than a pixel quantity threshold.

### Claim 15 (depends on 14)

The automatic parking system according to claim 14, wherein to generate multiple straight-line equations based on the multiple patches comprises:
perform, for each patch, the following operations in parallel:
get the patch;
obtain coordinates of one or more pixels with first color in the patch; and
fit a straight-line equation based on the obtained coordinates of the one or more pixels with first color.

### Claim 16 (depends on 15)

The automatic parking system according to claim 15, wherein the executable instructions further cause the one or more processors to:
perform, for each patch, the following operations in parallel:
determine whether a ratio of a quantity of pixels with first color matched by the straight-line equation to a quantity of all pixels with first color in the patch is greater than a ratio threshold, wherein a pixel with first color in the patch whose distance from a straight line represented by the straight-line equation is less than a distance threshold is considered to be a pixel with first color matched by the straight-line equation;
in response to the ratio being greater than the ratio threshold, transfer the straight-line equation to a coordinate system of the parking-line skeleton image to represent a straight line in the parking-line skeleton image, and take a quantity of pixels with first color matched by the straight-line equation in the parking-line skeleton image as a value of a comparison parameter; and
set the value of the comparison parameter to 0 in response to the ratio being equal to or less than the ratio threshold.

### Claim 17 (depends on 16)

The automatic parking system according to claim 16, wherein the executable instructions further cause the one or more processors to:
compare multiple comparison parameter values calculated for multiple patches; and
select the straight-line equation corresponding to a largest comparison parameter value among the multiple comparison parameter values as the optimal straight-line equation.

### Claim 18 (depends on 11)

The automatic parking system according to claim 11, wherein the executable instructions further cause the one or more processors to:
filter the at least one parking-line skeleton image to generate a filtered parking-line skeleton image.
