# US 10,769,476 B2 · License plate detection method and device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,769,476 B2
- Document ID: US10769476B2
- Status: Granted
- Kind: B2
- Application number: 16/086,789
- Filed: 2017-03-08
- Published: 2019-03-07
- Granted: 2020-09-08
- Earliest claimed: 2016-03-21
- Assignees: Hangzhou Hikvision Digital Technology Co., Ltd.
- Applicants: Hangzhou Hikvision Digital Technology Co., Ltd.
- Inventors: Shiliang Pu; Yi Niu; Zuozhou Pan; Binghua Luo
- Web page: https://patentgrep.com/patent/US10769476B2

## CPC

- G06V 20/63: {Scene text, e.g. street names}
- G06F 18/2155: {characterised by the incorporation of unlabelled data, e.g. multiple instance learning [MIL], semi-supervised techniques using expectation-maximisation [EM] or naïve labelling}
- G06F 18/2413: based on distances to training or reference patterns
- G06V 10/421: {by analysing segments intersecting the pattern}
- G06V 10/454: {Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]}
- G06V 10/46: Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features (colour feature extraction G06V10/56)
- G06V 10/764: using classification, e.g. of video objects
- G06V 20/62: Text, e.g. of license plates, overlay texts or captions on TV images
- G06V 20/625: {License plates}
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Embodiments of the present disclosure provide a method and apparatus for detecting a license plate. The method comprises: obtaining, according to pixel values of pixels in an image to be detected, a candidate license plate area M1 in the image to be detected (S101); calculating an aspect ratio of the candidate license plate area M1 and determining if the aspect ratio is greater than a first predefined threshold (S102); determining, if the aspect ratio is greater than the first predefined threshold, a new candidate license plate area M2 from the candidate license plate area M1 according to a predefined machine learning-based regression algorithm (S103); determining, according to a first predefined classification model, whether the candidate license plate area M2 is a license plate area (S104); determining, if the candidate license plate area M2 is a license plate area, the candidate license plate area M2 as a license plate area, and generating a detection result based on the candidate license plate area M2 (S105). The application of the solution improves the accuracy of license plate detection.

## Claims

### Claim 1

A method for detecting a license plate, comprising:
obtaining, according to pixel values of pixels in an image to be detected, a candidate license plate area M1 in the image to be detected;
calculating an aspect ratio of the candidate license plate area M1 and determining whether the aspect ratio is greater than a first predefined threshold;
if the aspect ratio is greater than the first predefined threshold, determining a new candidate license plate area M2 from the candidate license plate area M1 according to a predefined machine learning-based regression algorithm, wherein the candidate license plate area M2 is an area whose aspect ratio is no greater than the first predefined threshold;
determining, according to a first predefined classification model, whether the candidate license plate area M2 is a license plate area, wherein, the first predefined classification model is a classification model obtained by learning sample license plate areas through a machine learning algorithm, and wherein, in constructing of the first classification model, a lame amount of sample license plate areas previously found are learned through the machine learning algorithm so as to obtain a classification model, by means of which an area of the candidate license plate area is classified into a license plate area and non-license plate area;
if the candidate license plate area M2 is a license plate area, determining the candidate license plate area M2 as a license plate area, and generating a detection result based on the candidate license plate area M2,
wherein, determining a new candidate license plate area M2 from the candidate license plate area M1 according to a predefined machine learning-based regression algorithm comprises:
determining a position of a suspected character string in the candidate license plate area M1;
determining, through the predefined machine learning-based regression algorithm, a new boundary of the candidate license plate area according to the determined position; and
obtaining the candidate license plate area M2 according to the new boundary.

### Claim 2 (depends on 1)

The method of claim 1, wherein, obtaining, according to pixel values of pixels in an image to be detected, a candidate license plate area M1 in the image to be detected comprises:
obtaining a valid pixel segment in each pixel row of the image to be detected in a predefined scan order, wherein, the valid pixel segment is a pixel segment that is determined according to pixels having a greyscale jump value greater than a second predefined threshold in a pixel row;
calculating a boundary similarity for vertically adjacent valid pixel segments, according to pixel values of pixels at both ends of each valid pixel segment and pixel values of pixels at both ends of another valid pixel segment vertically adjacent to that valid pixel segment;
merging adjacent valid pixel segments which have a boundary similarity greater than a third predefined threshold; and
obtaining the candidate license plate area M1 in the image to be detected according to the merged valid pixel segments.

### Claim 3 (depends on 2)

The method of claim 2, wherein, obtaining a valid pixel segment in each pixel row of the image to be detected in a predefined scan order comprises:
obtaining a valid pixel segment in each pixel row of the image to be detected in a predefined scan order by:
calculating a greyscale jump value of each pixel in a pixel row X, wherein the pixel row X is any pixel row in the image to be detected;
selecting pixels having a greyscale value greater than the second predefined threshold;
obtaining a candidate pixel segment in the pixel row X according to a pixel having a maximum horizonal coordinate and a pixel having a minimum horizonal coordinate among the selected pixels;
determining whether greyscale jump values of pixels in the candidate pixel segment conform to a predefined greyscale jumping rule; and
if so, determining the candidate pixel segment as a valid pixel segment.

### Claim 4 (depends on 2)

The method of claim 2, wherein, obtaining the candidate license plate area M1 in the image to be detected according to the merged valid pixel segments comprises:
determining a suspected character string area in the merged valid pixel segments;
obtaining color information of character string according to a pixel value of a pixel in the suspected character string area, and obtaining color information of background according to a pixel value of a pixel not in the non-suspected character string area in the merged valid pixel segments;
determining boundary of the candidate license plate M1 according to the color information of character string and the color information of background; and
obtaining the candidate license plate area M1 according to the determined boundary.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
determining, in response to a determination that the candidate license plate area M2 is not a license plate area, whether brightness of the candidate license plate area M2 is within a predefined range of brightness;
performing, if the brightness of the license plate candidate area M2 is not within a predefined range of brightness, grey equalization on the candidate license plate area M2;
determining whether the grey-equalized candidate license plate area M2 is a license plate area according to a second predefined classification model, wherein, the second predefined classification model is a classification model obtained by learning grey-equalized sample license plate areas through a machine learning algorithm;
performing, if the candidate license plate area M2 is a license plate area, the steps of determining the candidate license plate area M2 as a license plate area and generating a detection result based on the candidate license plate area M2.

### Claim 6 (depends on 1)

A non-transitory computer-readable storage medium, which is used for storing executable program codes, wherein the executable program codes is configured for, when being executed, implement the method for detecting a license plate according to claim 1.

### Claim 7

A terminal, comprising: a housing, a processor, a memory, a circuit board and a power supply circuit, wherein, the circuit board is arranged within a space enclosed by the housing; the processor and memory are disposed on the circuit board; the power supply circuit is configured for supplying power to circuits and devices of the terminal; the memory is configured for storing executable program instructions; and the processor is configured for executing the executable program instructions stored in the memory to perform:
obtaining, according to pixel values of pixels in an image to be detected, a candidate license plate area M1 in the image to be detected;
calculating an aspect ratio of the candidate license plate area M1 and determining whether the aspect ratio is greater than a first predefined threshold;
if the aspect ratio is greater than the first predefined threshold, determining a new candidate license plate area M2 from the candidate license plate area M1 according to a predefined machine learning-based regression algorithm, wherein the candidate license plate area M2 is an area whose aspect ratio is no greater than the first predefined threshold;
determining, according to a first predefined classification model, whether the candidate license plate area M2 is a license plate area, wherein, the first predefined classification model is a classification model obtained by learning sample license plate areas through a machine learning algorithm, and wherein, in constructing of the first classification model, a lame amount of sample license plate areas previously found are learned through the machine learning algorithm so as to obtain a classification model, by means of which an area of the candidate license plate area is classified into a license plate area and non-license plate area;
if the candidate license plate area M2 is a license plate area, determining the candidate license plate area M2 as a license plate area, and generating a detection result based on the candidate license plate area M2,
wherein, determining a new candidate license plate area M2 from the candidate license plate area M1 according to a predefined machine learning-based regression algorithm comprises:
determining a position of a suspected character string in the candidate license plate area M1;
determining, through the predefined machine learning-based regression algorithm, a new boundary of the candidate license plate area according to the determined position; and
obtaining the candidate license plate area M2 according to the new boundary.

### Claim 8 (depends on 7)

The terminal of claim 7, wherein, obtaining, according to pixel values of pixels in an image to be detected, a candidate license plate area M1 in the image to be detected comprises:
obtaining a valid pixel segment in each pixel row of the image to be detected in a predefined scan order, wherein, the valid pixel segment is a pixel segment that is determined according to pixels having a greyscale jump value greater than a second predefined threshold in a pixel row;
calculating a boundary similarity for vertically adjacent valid pixel segments, according to pixel values of pixels at both ends of each valid pixel segment and pixel values of pixels at both ends of another valid pixel segment vertically adjacent to that valid pixel segment;
merging adjacent valid pixel segments which have a boundary similarity greater than a third predefined threshold; and
obtaining the candidate license plate area M1 in the image to be detected according to the merged valid pixel segments.

### Claim 9 (depends on 8)

The terminal of claim 8, wherein, obtaining a valid pixel segment in each pixel row of the image to be detected in a predefined scan order comprises:
obtaining a valid pixel segment in each pixel row of the image to be detected in a predefined scan order by:
calculating a greyscale jump value of each pixel in a pixel row X, wherein the pixel row X is any pixel row in the image to be detected;
selecting pixels having a greyscale value greater than the second predefined threshold;
obtaining a candidate pixel segment in the pixel row X according to a pixel having a maximum horizonal coordinate and a pixel having a minimum horizonal coordinate among the selected pixels;
determining whether greyscale jump values of pixels in the candidate pixel segment conform to a predefined greyscale jumping rule; and
if so, determining the candidate pixel segment as a valid pixel segment.

### Claim 10 (depends on 8)

The terminal of claim 8, wherein, obtaining the candidate license plate area M1 in the image to be detected according to the merged valid pixel segments comprises:
determining a suspected character string area in the merged valid pixel segments;
obtaining color information of character string according to a pixel value of a pixel in the suspected character string area, and obtaining color information of background according to a pixel value of a pixel not in the non-suspected character string area in the merged valid pixel segments;
determining boundary of the candidate license plate M1 according to the color information of character string and the color information of background; and
obtaining the candidate license plate area M1 according to the determined boundary.

### Claim 11 (depends on 7)

The terminal of claim 7, wherein, determining a new candidate license plate area M2 from the candidate license plate area M1 according to a predefined machine learning-based regression algorithm comprises:
determining a position of a suspected character string in the candidate license plate area M1;
determining, through the predefined machine learning-based regression algorithm, a new boundary of the candidate license plate area according to the determined position; and
obtaining the candidate license plate area M2 according to the new boundary.

### Claim 12 (depends on 7)

The terminal of claim 7, the processor is further configured for executing the executable program instructions stored in the memory to perform:
determining, in response to a determination that the candidate license plate area M2 is not a license plate area, whether brightness of the candidate license plate area M2 is within a predefined range of brightness;
performing, if the brightness of the license plate candidate area M2 is not within a predefined range of brightness, grey equalization on the candidate license plate area M2;
determining whether the grey-equalized candidate license plate area M2 is a license plate area according to a second predefined classification model, wherein, the second predefined classification model is a classification model obtained by learning grey-equalized sample license plate areas through a machine learning algorithm;
performing, if the candidate license plate area M2 is a license plate area, the steps of determining the candidate license plate area M2 as a license plate area and generating a detection result based on the candidate license plate area M2.