# US 2017/0372161 A1 · Intelligent automatic license plate recognition for electronic tolling environments

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2017/0372161 A1
- Document ID: US20170372161A1
- Status: Published application
- Kind: A1
- Application number: 15/254,859
- Filed: 2016-09-01
- Published: 2017-12-28
- Earliest claimed: 2016-06-24
- Assignees: Accenture Global Solutions Limited
- Applicants: Accenture Global Solutions Limited
- Inventors: Ricardo André Santos Gomes de Almeida; António Ricardo Ruano Pinto; Romeu Rodrigues Figueira
- Web page: https://patentgrep.com/patent/US20170372161A1

## CPC

- G06V 20/54: of traffic, e.g. cars on the road, trains or boats
- G06F 18/256: {of results relating to different input data, e.g. multimodal recognition}
- G06V 10/811: {the classifiers operating on different input data, e.g. multi-modal recognition}
- G06V 20/62: Text, e.g. of license plates, overlay texts or captions on TV images
- G06V 20/63: {Scene text, e.g. street names}
- G07C 9/20: involving the use of a pass
- G06V 20/625: {License plates}
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G07C: TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE

## Abstract

An intelligent automatic license plate recognition (IALPR) system implements technical solutions that improve the accuracy of automatic license plate recognition. The IALPR analyzes an image of a vehicle proximate to a toll collection point using optical character recognition (OCR), and determines candidate license plate identifications based, at least in part, on the corresponding OCR confidence level. The IALPR can also perform fingerprinting for candidate license plate images and matching analysis with a knowledge base, resulting in additional confidence levels. The IALPR can also perform behavioral analysis on the candidate license plate identifications, including trip context analysis, historical behavioral analysis, or other analytics. The IALPR can generate an overall confidence level for the candidate license plate identifications responsive to the OCR and vehicle fingerprint confidence levels and the behavioral analysis. This enhanced analysis helps the IALPR reduce the number of incorrect license plate identifications and reduce the need for human review.

## Claims

### Claim 1

A method comprising:
receiving, at a processor, an image including a representation of a license plate of a vehicle when the vehicle is proximate to a current toll collection point;
performing, by the processor, a first optical character recognition (OCR) process on the image to determine a candidate license plate string;
determining, by the processor, a candidate license plate identification associated with the candidate license plate string;
determining, by the processor, an OCR confidence level for the candidate license plate identification based on the first OCR process;
performing, by the processor, a behavioral analysis of geospatial vehicle position data related to the candidate license plate identification;
generating, by the processor, a behavioral analysis result based on the analysis of the geospatial vehicle position data; and
determining, by the processor, an overall confidence level for the candidate license plate identification responsive to the OCR confidence level and the behavioral analysis result.

### Claim 2 (depends on 1)

The method of claim 1, where:
performing the behavioral analysis comprises performing a trip context analysis, the trip context analysis comprising:
searching for a related toll event occurring at a related toll collection point, the related toll collection point with respect to the current toll collection point, and having a license plate identification matching the candidate license plate identification; and
increasing the overall confidence level for the candidate license plate identification when:
the related toll event is found, and
the related toll event occurred within a travel time range,
the travel time range based on an average travel time between the current toll collection point and the related toll collection point.

### Claim 3 (depends on 1)

The method of claim 1 where:
performing the behavioral analysis comprises performing a historical behavioral analysis, the historical behavioral analysis comprising:
searching for historical toll events occurring at the current toll collection point, the historical toll events associated with a license plate identification matching the candidate license plate identification;
determining, by the processor, a historical daily toll event pattern associated with the license plate identification, the determining responsive to the historical toll events;
determining, by the processor, whether a toll event corresponding to the image occurred within a daily toll time range of the historical daily toll event pattern for the license plate identification; and
increasing the overall confidence level for the candidate license plate identification when the toll event corresponding to the image occurred within the daily toll time range.

### Claim 4 (depends on 1)

The method of claim 1 further comprising:
determining, by the processor, a physical vehicle attribute of the vehicle in the image;
comparing, by the processor, the physical vehicle attribute to a pre-determined vehicle attribute associated with the candidate license plate identification; and
increasing the overall confidence level for the candidate license plate identification when the physical vehicle attribute matches the pre-determined vehicle attribute.

### Claim 5 (depends on 1)

The method of claim 1 further comprising determining the overall confidence level using a statistical model of random variables.

### Claim 6 (depends on 1)

The method of claim 1 further comprising:
performing, by the processor, a second OCR process on the image to determine a second candidate license plate string, the second OCR process different from the first OCR process;
determining, by the processor, a second candidate license plate identification associated with the second candidate license plate string;
determining, by the processor, a second OCR confidence level for the second candidate license plate identification based on the second OCR process; and
determining, by the processor, the overall confidence level for the candidate license plate identification based on the OCR confidence level and the second OCR confidence level.

### Claim 7 (depends on 1)

The method of claim 1 further comprising:
performing, by the processor, a fingerprint process on the license plate of the image to determine a current license plate fingerprint;
comparing, by the processor, the current license plate fingerprint to stored license plate fingerprints;
determining, by the processor, a closest license plate fingerprint to the current license plate fingerprint, the determining responsive to the comparing;
determining, by the processor, a fingerprint candidate license plate identification, the fingerprint candidate license plate identification responsive to the closest license plate fingerprint; and
determining, by the processor, a fingerprint confidence level for the fingerprint candidate license plate identification responsive to the closest license plate fingerprint.

### Claim 8 (depends on 7)

The method of claim 7 further comprising:
determining an intermediate confidence level for the candidate license plate identification, the determining responsive to the OCR confidence level and the fingerprint confidence level; and
determining, by the processor, that the intermediate confidence level does not exceed an intermediate confidence level threshold.

### Claim 9 (depends on 1)

The method of claim 1 further comprising:
determining, by the processor, that the overall confidence level does not exceed a minimum confidence threshold;
displaying the image at the toll collection point on a graphical user interface in communication with the processor;
receiving, from the graphical user interface, an external license plate identification for the image; and
updating a license plate database with the external license plate identification.

### Claim 10 (depends on 1)

The method of claim 1 further comprising:
generating, by the processor, a derivative candidate license plate string from the candidate license plate string;
determining, by the processor, a derivative candidate license plate identification responsive to the derivative candidate license plate string;
determining, by the processor, a derivative confidence level for the derivative candidate license plate identification;
performing, by the processor, the behavioral analysis on the derivative candidate license plate identification; and
determining, by the processor, a derivative overall confidence level for the derivative candidate license plate identification responsive to the derivative confidence level and the behavioral analysis.

### Claim 11

A system comprising:
a communication interface configured to receive:
a candidate license plate identification associated with an image of a vehicle proximate to a current toll collection point during a current toll event, and
an identification confidence level associated with the candidate license plate identification;
behavioral analysis circuitry coupled to the communication interface, the behavioral analysis circuitry configured to perform a behavioral analysis of geospatial vehicle positioning data related to the candidate license plate identification to generate behavioral information; and
probability evaluation circuitry configured to determine an overall confidence level associated with the candidate license plate identification responsive to the candidate license plate identification, the identification confidence level, and the behavioral information.

### Claim 12 (depends on 11)

The system of claim 11 wherein the behavioral analysis circuitry is further configured to:
search for a related toll event occurring at a related toll collection point with respect to the current toll collection point and having a license plate identification matching the candidate license plate identification;
determine a current travel time from a time difference between the current toll event and the related toll event;
determine an average travel time between the current toll collection point and the related toll collection point; and
determine a degree of difference between the current travel time and the average travel time.

### Claim 13 (depends on 11)

The system of claim 11 wherein the behavioral analysis circuitry is further configured to:
search for historical toll events occurring at the current toll collection point with a license plate identification matching the candidate license plate identification;
determine a historical daily toll event pattern including a daily toll time range for the license plate identification responsive to the historical toll events; and
determine whether the current toll event occurred within the daily toll time range.

### Claim 14 (depends on 11)

The system of claim 11 wherein the communication interface is further configured to receive a physical vehicle attribute of the vehicle in the image;
the system further comprising vehicle attribute analysis circuitry configured to:
determine whether the physical vehicle attribute matches a pre-determined vehicle attribute associated with a license plate identification matching the candidate license plate identification; and
determine a physical vehicle attribute matching confidence level; and
the probability evaluation circuitry further configured to alter the overall confidence level for the candidate license plate identification responsive to determining that the physical vehicle attribute matches the pre-determined vehicle attribute and responsive to the physical vehicle attribute matching confidence level.

### Claim 15 (depends on 11)

The system of claim 11 wherein the probability evaluation circuitry further comprises a statistical model of random variables.

### Claim 16

A computer-readable medium including instructions that, when executed by circuitry elements, cause the circuitry elements to perform a method comprising:
receiving, via a communication interface, a candidate license plate identification associated with an image of a vehicle proximate to a current toll collection point during a current toll event and an identification confidence level associated with the candidate license plate identification;
performing, by behavioral analysis circuitry coupled to the communication interface, a behavioral analysis process on the candidate license plate identification to generate behavioral information; and
determining, by probability evaluation circuitry coupled to the behavioral analysis circuitry, an overall confidence level associated with the candidate license plate identification responsive to the candidate license plate identification, the identification confidence level, and the behavioral information.

### Claim 17 (depends on 16)

The computer-readable medium of claim 16, wherein the method further comprises:
searching, by the behavioral analysis circuitry, for a related toll event occurring at a related toll collection point with respect to the current toll collection point and having a license plate identification matching the candidate license plate identification;
determining, by the behavioral analysis circuitry, a current travel time from a time difference between the current toll event and the related toll event;
determining, by the behavioral analysis circuitry, an average travel time between the current toll collection point and the related toll collection point;
determining, by the behavioral analysis circuitry, a degree of difference between the current travel time and the average travel time; and
altering, by the probability evaluation circuitry, the overall confidence level for the candidate license plate identification responsive to finding the related toll event and responsive to determining the degree of difference between the current travel time and the average travel time.

### Claim 18 (depends on 16)

The computer-readable medium of claim 16, wherein the method further comprises:
searching, by the behavioral analysis circuitry, for historical toll events occurring at the current toll collection point with a license plate identification matching the candidate license plate identification;
determining, by the behavioral analysis circuitry, a historical daily toll event pattern including a daily toll time range for the license plate identification responsive to the historical toll events;
determining, by the behavioral analysis circuitry, whether the current toll event occurred within the daily toll time range; and
altering, by the probability evaluation circuitry, the overall confidence level for the candidate license plate identification responsive to determining whether the current toll event occurred within the daily toll time range.

### Claim 19 (depends on 16)

The computer-readable medium of claim 16, wherein the method further comprises:
receiving, by the communication interface, a physical vehicle attribute of the vehicle in the image;
determining, by vehicle attribute analysis circuitry, whether the physical vehicle attribute matches a pre-determined vehicle attribute associated with the candidate license plate identification;
determining, by the vehicle attribute analysis circuitry, a physical vehicle attribute matching confidence level; and
altering, by the probability evaluation circuitry, the overall confidence level for the candidate license plate identification responsive to determining that the physical vehicle attribute matches the pre-determined vehicle attribute and responsive to determining the physical vehicle attribute matching confidence level.

### Claim 20 (depends on 16)

The computer-readable medium of claim 16, wherein the method further comprises determining, by the probability evaluation circuitry, the overall confidence level associated with the candidate license plate identification using a statistical model of random variables.
