# US 11,113,582 B2 · Method and system for facilitating detection and identification of vehicle parts

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,113,582 B2
- Document ID: US11113582B2
- Status: Granted
- Kind: B2
- Application number: 16/555,955
- Filed: 2019-08-29
- Published: 2020-03-05
- Granted: 2021-09-07
- Earliest claimed: 2018-08-31
- Assignees: Advanced New Technologies Co., Ltd.
- Applicants: Alibaba Group Holding Limited
- Inventors: Jian Wang; Danni Cheng
- Web page: https://patentgrep.com/patent/US11113582B2

## CPC

- G06Q 10/10: Office automation; Time management
- G06F 18/2431: Multiple classes
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06T 7/74: {involving reference images or patches}
- G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI]
- G06V 10/764: using classification, e.g. of video objects
- G07C 5/008: {communicating information to a remotely located station (transmission systems for measured values G08C)}
- G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30164: Workpiece; Machine component
- G06T 2207/30252: Vehicle exterior; Vicinity of vehicle
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G07C: TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE

## Abstract

One embodiment facilitates detection and identification of vehicle parts. During operation, the system stores a captured image of a vehicle, wherein the captured image includes a plurality of parts of the vehicle. The system detects a first area of the captured image in which a first part of the vehicle is located. The system identifies, from a plurality of predetermined classes, a class corresponding to the first part in the detected first area, wherein the class indicates a location of the first part in relation to the vehicle. The system generates a result which indicates a list including an insurance claim item and corresponding damages based on the first area, the first part, and the class corresponding to the first part.

## Claims

### Claim 1

A computer-implemented method for facilitating detection and identification of vehicle parts, the method comprising:
storing a captured image of a vehicle, wherein the captured image includes a plurality of parts of the vehicle;
training a first algorithm based on at least images with pre-marked areas indicating final locations for multiple parts of the vehicle;
detecting, based on the trained first algorithm, a first area of the captured image in which a first part of the vehicle is located;
identifying, from a plurality of predetermined classes, a class corresponding to the first part in the detected first area, wherein the class indicates a location of the first part in relation to the vehicle; and
generating a result which indicates a list including an insurance claim item and corresponding damages based on the first area, the first part, and the class corresponding to the first part.

### Claim 2 (depends on 1)

The method of claim 1, wherein detecting the first area comprises:
dividing the captured image into a grid of squares;
mapping, for a respective square, the respective square to a first number of target areas, wherein a respective target area comprises a bounding box associated with at least the respective square;
determining, for the respective target area, a confidence coefficient which indicates a likelihood that the respective target area corresponds to a first predetermined class;
determining, for the respective square, a probability which indicates a likelihood that the respective square corresponds to each of the predetermined classes; and
selecting, based on the confidence coefficient and the probability, a second area as the first area in which the first part of the vehicle is located, wherein the second area is selected from all target areas mapped to each of the squares.

### Claim 3 (depends on 2)

The method of claim 2, wherein a second number of squares of the grid in a horizontal direction are the same as a second number of squares of the grid in a vertical direction, and wherein the first number and the second number are positive integers.

### Claim 4 (depends on 1)

The method of claim 1, wherein prior to storing the captured image of the vehicle, the method further comprises:
acquiring a plurality of unsorted images of the vehicle, wherein the unsorted images are captured directly by a user of the vehicle; and
sorting the unsorted images by removing images which are irrelevant or abnormal to obtain a set of captured images which include the captured image.

### Claim 5 (depends on 1)

The method of claim 1,
wherein training the first algorithm is further based on:
a plurality of sample images of a predetermined class corresponding to a respective part of the vehicle.

### Claim 6 (depends on 1)

The method of claim 1, wherein the first algorithm is a You Only Look Once v2 (yolov2) algorithm, and wherein the first algorithm further comprises a model based on a neural network.

### Claim 7 (depends on 1)

The method of claim 1, wherein a class includes one or more of:
a left front door;
a right front door;
a left rear door;
a right rear door;
a front bumper;
a rear bumper;
a left front wheel fender;
a right front wheel fender;
a left rear wheel fender;
a right rear wheel fender;
a left front tire;
a right front tire;
a left rear tire;
a right rear tire;
a lower grille;
a middle grille;
a left headlight;
a right headlight;
a left brake light;
a right brake light;
a left front emergency light;
a right front emergency light;
a left rear emergency light;
a right rear emergency light;
a number plate;
a front engine hood; and
a trunk.

### Claim 8

A computer system for facilitating detection and identification of vehicle parts, the system comprising:
a processor; and
a memory coupled to the processor and storing instructions, which when executed by the processor cause the processor to perform a method, wherein the computer system is a storage device, the method comprising:
storing a captured image of a vehicle, wherein the captured image includes a plurality of parts of the vehicle;
training a first algorithm based on at least images with pre-marked areas indicating final locations for multiple parts of the vehicle;
detecting, based on the trained first algorithm, a first area of the captured image in which a first part of the vehicle is located;
identifying, from a plurality of predetermined classes, a class corresponding to the first part in the detected first area, wherein the class indicates a location of the first part in relation to the vehicle; and
generating a result which indicates a list including an insurance claim item and corresponding damages based on the first area, the first part, and the class corresponding to the first part.

### Claim 9 (depends on 8)

The computer system of claim 8, wherein detecting the first area comprises:
dividing the captured image into a grid of squares;
mapping, for a respective square, the respective square to a first number of target areas, wherein a respective target area comprises a bounding box associated with at least the respective square;
determining, for the respective target area, a confidence coefficient which indicates a likelihood that the respective target area corresponds to a first predetermined class;
determining, for the respective square, a probability which indicates a likelihood that the respective square corresponds to each of the predetermined classes; and
selecting, based on the confidence coefficient and the probability, a second area as the first area in which the first part of the vehicle is located, wherein the second area is selected from all target areas mapped to each of the squares.

### Claim 10 (depends on 9)

The computer system of claim 9, wherein a second number of squares of the grid in a horizontal direction are the same as a second number of squares of the grid in a vertical direction, and wherein the first number and the second number are positive integers.

### Claim 11 (depends on 8)

The computer system of claim 8, wherein prior to storing the captured image of the vehicle, the method further comprises:
acquiring a plurality of unsorted images of the vehicle, wherein the unsorted images are captured directly by a user of the vehicle; and
sorting the unsorted images by removing images which are irrelevant or abnormal to obtain a set of captured images which include the captured image.

### Claim 12 (depends on 8)

The computer system of claim 8,
wherein training the first algorithm is further based on:
a plurality of sample images of a predetermined class corresponding to a respective part of the vehicle.

### Claim 13 (depends on 8)

The computer system of claim 8, wherein the first algorithm is a You Only Look Once v2 (yolov2) algorithm, and wherein the first algorithm further comprises a model based on a neural network.

### Claim 14 (depends on 8)

The computer system of claim 8, wherein a class includes one or more of:
a left front door;
a right front door;
a left rear door;
a right rear door;
a front bumper;
a rear bumper;
a left front wheel fender;
a right front wheel fender;
a left rear wheel fender;
a right rear wheel fender;
a left front tire;
a right front tire;
a left rear tire;
a right rear tire;
a lower grille;
a middle grille;
a left headlight;
a right headlight;
a left brake light;
a right brake light;
a left front emergency light;
a right front emergency light;
a left rear emergency light;
a right rear emergency light;
a number plate;
a front engine hood; and
a trunk.

### Claim 15

A non-transitory computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform a method for facilitating detection and identification of vehicle parts, the method comprising:
storing a captured image of a vehicle, wherein the captured image includes a plurality of parts of the vehicle;
training a first algorithm based on at least images with pre-marked areas indicating final locations for multiple parts of the vehicle;
detecting, based on the trained first algorithm, a first area of the captured image in which a first part of the vehicle is located;
identifying, from a plurality of predetermined classes, a class corresponding to the first part in the detected first area, wherein the class indicates a location of the first part in relation to the vehicle; and
generating a result which indicates a list including an insurance claim item and corresponding damages based on the first area, the first part, and the class corresponding to the first part.

### Claim 16 (depends on 15)

The storage medium of claim 15, wherein prior to storing the captured image of the vehicle, the method further comprises:
acquiring a plurality of unsorted images of the vehicle, wherein the unsorted images are captured directly by a user of the vehicle; and
sorting the unsorted images by removing images which are irrelevant or abnormal to obtain a set of captured images which include the captured image.

### Claim 17 (depends on 15)

The storage medium of claim 15,
wherein training the first algorithm is further based on:
a plurality of sample images of a predetermined class corresponding to a respective part of the vehicle.

### Claim 18 (depends on 15)

The storage medium of claim 15, wherein a class includes one or more of:
a left front door;
a right front door;
a left rear door;
a right rear door;
a front bumper;
a rear bumper;
a left front wheel fender;
a right front wheel fender;
a left rear wheel fender;
a right rear wheel fender;
a left front tire;
a right front tire;
a left rear tire;
a right rear tire;
a lower grille;
a middle grille;
a left headlight;
a right headlight;
a left brake light;
a right brake light;
a left front emergency light;
a right front emergency light;
a left rear emergency light;
a right rear emergency light;
a number plate;
a front engine hood; and
a trunk.

### Claim 19 (depends on 15)

The storage medium of claim 15, wherein detecting the first area comprises:
dividing the captured image into a grid of squares;
mapping, for a respective square, the respective square to a first number of target areas, wherein a respective target area comprises a bounding box associated with at least the respective square;
determining, for the respective target area, a confidence coefficient which indicates a likelihood that the respective target area corresponds to a first predetermined class;
determining, for the respective square, a probability which indicates a likelihood that the respective square corresponds to each of the predetermined classes; and
selecting, based on the confidence coefficient and the probability, a second area as the first area in which the first part of the vehicle is located, wherein the second area is selected from all target areas mapped to each of the squares.

### Claim 20 (depends on 19)

The storage medium of claim 19, wherein a second number of squares of the grid in a horizontal direction are the same as a second number of squares of the grid in a vertical direction, and wherein the first number and the second number are positive integers.