# US 11,443,288 B2 · Automatic assessment of damage and repair costs in vehicles

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,443,288 B2
- Document ID: US11443288B2
- Status: Granted
- Kind: B2
- Application number: 16/871,346
- Filed: 2020-05-11
- Published: 2020-08-27
- Granted: 2022-09-13
- Earliest claimed: 2016-04-06
- Assignees: American International Group, Inc.
- Applicants: American International Group, Inc.
- Inventors: Vikas Taliwal; Siddhartha Dalal; Kaigang Li; Gaurav Sharma
- Web page: https://patentgrep.com/patent/US11443288B2

## CPC

- G06Q 10/20: Administration of product repair or maintenance
- G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence)
- G06F 18/22: Matching criteria, e.g. proximity measures
- G06Q 30/0283: Price estimation or determination
- G06Q 40/08: Insurance
- G06T 7/0004: {Industrial image inspection}
- G06T 7/11: Region-based segmentation
- G06T 7/143: involving probabilistic approaches, e.g. Markov random field [MRF] modelling
- G06T 7/33: using feature-based methods
- G06T 7/40: Analysis of texture (depth or shape recovery from texture G06T7/529)
- G06T 19/003: {Navigation within 3D models or images}
- G06V 10/443: {by matching or filtering}
- G06V 10/462: {Salient features, e.g. scale invariant feature transforms [SIFT]}
- G06V 10/75: Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V 10/82: using neural networks
- G06V 20/20: in augmented reality scenes
- H04W 4/02: Services making use of location information
- G06F 2203/04802: 3D-info-object: information is displayed on the internal or external surface of a three dimensional manipulable object, e.g. on the faces of a cube that can be rotated by the user
- G06T 7/194: involving foreground-background segmentation
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30156: Vehicle coating
- G06T 2207/30252: Vehicle exterior; Vicinity of vehicle
- G06V 2201/08: Detecting or categorising vehicles
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72)

## Abstract

A system and method are provided for automatically estimating a repair cost for a vehicle. A method includes: receiving, at a server computing device over an electronic network, one or more images of a damaged vehicle from a client computing device; performing image processing operations on each of the one or more images to detect external damage to a first set of parts of the vehicle; inferring internal damage to a second set of parts of the vehicle based on the detected external damage; and, calculating an estimated repair cost for the vehicle based on the detected external damage and inferred internal damage based on accessing a parts database that includes repair and labor costs for each part in the first and second sets of parts.

## Claims

### Claim 1

A method, comprising:
causing a displaying of an outline of a selected damaged part of a damaged vehicle to be captured with a camera of a client device;
causing the camera of the client device to capture an image of the damaged vehicle based on the displaying of the outline of the selected damaged part;
receiving, at a server computing device over an electronic network and from the client device, the image of the damaged vehicle;
aligning the image to an undamaged version of the damaged vehicle;
segmenting the image into vehicle parts; and
detecting damage to a set of parts of the damaged vehicle by comparing portions of each vehicle part in the image to corresponding portions of each vehicle part in the undamaged version of the damaged vehicle, wherein detecting damage to the set of parts includes:
comparing at least one of edge distribution, texture comparison, and spatial correlation of portions of each vehicle part in the image to corresponding portions of each vehicle part in the undamaged version of the damaged vehicle;
determining whether at least one of the edge distribution, the texture comparison, and the spatial correlation exceeds a respective threshold difference value, wherein damage is detected in a portion of a vehicle part in the image if at least one of the edge distribution, the texture comparison, and the spatial correlation exceed the respective threshold difference value;
detecting a pose of the damaged vehicle in the image; and
determining which external vehicle parts are damaged in the image; and
calculating an estimated repair cost for the damaged vehicle based on which external vehicle parts are damaged based on accessing a parts database that includes repair costs.

### Claim 2 (depends on 1)

The method of claim 1, wherein the parts database that includes repair costs includes estimates for parts and labor for individual parts.

### Claim 3 (depends on 1)

The method of claim 1, further comprising removing artifacts from the image by:
removing background material from the image; and
removing specular reflection due to incident light on the damaged vehicle shown in the image.

### Claim 4 (depends on 1)

The method of claim 1, wherein damage is detected in a portion of a vehicle part in the image if at least two of the edge distribution, the texture comparison, and the spatial correlation exceed the respective threshold difference value.

### Claim 5 (depends on 1)

The method of claim 1, wherein the detecting damage to the set of parts includes comparing each of edge distribution, texture comparison, and spatial correlation of portions of each vehicle part in the image to corresponding portions of each vehicle part in the undamaged version of the damaged vehicle.

### Claim 6 (depends on 1)

The method of claim 1, wherein the detecting the pose of the damaged vehicle in the image comprises:
training a first Convolutional Neural Networks (CNN) of a plurality of CNNs to detect the pose of a damaged vehicle in the image; and
training each of the plurality of CNNs to detect damage on a respective vehicle part of a plurality of external vehicle parts; and
executing the first CNN to detect the pose of the damaged vehicle in the image.

### Claim 7 (depends on 1)

The method of claim 1, wherein detecting damage to the set of parts further includes inferring damage to internal parts of the damaged vehicle from the determined damaged external vehicle parts; and
wherein calculating the estimated repair cost for the damaged vehicle is further based on which internal vehicle parts are inferred to be damaged.

### Claim 8 (depends on 7)

The method of claim 7, wherein inferring damage to internal parts of the damaged vehicle from the determined damaged external vehicle parts comprises executing a Markov Random Field (MRF) algorithm.

### Claim 9

A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause a computing device to perform operations of:
causing a displaying of an outline of a selected damaged part of a damaged vehicle to be captured with a camera of a client device;
causing the camera of the client device to capture an image of the damaged vehicle based on the displaying of the outline of the selected damaged part;
receiving, from the client device, the image of the damaged vehicle;
aligning the image to an undamaged version of the damaged vehicle;
segmenting the image into vehicle parts; and
detecting damage to a set of parts of the damaged vehicle by comparing portions of each vehicle part in the image to corresponding portions of each vehicle part in the undamaged version of the damaged vehicle, wherein detecting damage to the set of parts includes:
comparing at least one of edge distribution, texture comparison, and spatial correlation of portions of each vehicle part in the image to corresponding portions of each vehicle part in the undamaged version of the damaged vehicle;
determining whether at least one of the edge distribution, the texture comparison, and the spatial correlation exceeds a respective threshold difference value, wherein damage is detected in a portion of a vehicle part in the image if at least one of the edge distribution, the texture comparison, and the spatial correlation exceed the respective threshold difference value;
detecting a pose of the damaged vehicle in the image; and
determining which external vehicle parts are damaged in the image; and
calculating an estimated repair cost for the damaged vehicle based on which external vehicle parts are damaged based on accessing a parts database that includes repair costs.

### Claim 10 (depends on 9)

The computer-readable medium of claim 9, wherein the parts database that includes repair costs includes estimates for parts and labor for individual parts.

### Claim 11 (depends on 9)

The computer-readable medium of claim 9, further comprising removing artifacts from the image by:
removing background material from the image; and
removing specular reflection due to incident light on the damaged vehicle shown in the image.

### Claim 12 (depends on 9)

The computer-readable medium of claim 9, wherein damage is detected in a portion of a vehicle part in the image if at least two of the edge distribution, the texture comparison, and the spatial correlation exceed the respective threshold difference value.

### Claim 13 (depends on 9)

The computer-readable medium of claim 9, wherein the detecting damage to the set of parts includes comparing each of edge distribution, texture comparison, and spatial correlation of portions of each vehicle part in the image to corresponding portions of each vehicle part in the undamaged version of the damaged vehicle.

### Claim 14 (depends on 9)

The computer-readable medium of claim 9, wherein the detecting the pose of the damaged vehicle in the image comprises:
training a first Convolutional Neural Networks (CNN) of a plurality of CNNs to detect the pose of a damaged vehicle in the image; and
training each of the plurality of CNNs to detect damage on a respective vehicle part of a plurality of external vehicle parts; and
executing the first CNN to detect the pose of the damaged vehicle in the image.

### Claim 15 (depends on 9)

The computer-readable medium of claim 9, wherein detecting damage to the set of parts further includes inferring damage to internal parts of the damaged vehicle from the determined damaged external vehicle parts; and wherein calculating the estimated repair cost for the damaged vehicle is further based on which internal vehicle parts are inferred to be damaged.

### Claim 16 (depends on 15)

The computer-readable medium of claim 15, wherein the inferring damage to internal parts of the damaged vehicle from the determined damaged external vehicle parts comprises executing a Markov Random Field (MRF) algorithm.

### Claim 17

A computing device, comprising: a processor; and
a memory storing instructions that, when executed by the processor, cause the computing device to perform operations of:
causing a displaying of an outline of a selected damaged part of a damaged vehicle to be captured with a camera of a client device;
causing the camera of the client device to capture an image of the damaged vehicle based on the displaying of the outline of the selected damaged part;
receiving, from the client device, the image of the damaged vehicle;
aligning the image to an undamaged version of the damaged vehicle;
segmenting the image into vehicle parts; and
detecting damage to a set of parts of the damaged vehicle by comparing portions of each vehicle part in the image to corresponding portions of each vehicle part in the undamaged version of the damaged vehicle, wherein detecting damage to the set of parts includes:
comparing at least one of edge distribution, texture comparison, and spatial correlation of portions of each vehicle part in the image to corresponding portions of each vehicle part in the undamaged version of the damaged vehicle;
determining whether at least one of the edge distribution, the texture comparison, and the spatial correlation exceeds a respective threshold difference value, wherein damage is detected in a portion of a vehicle part in the image if at least one of the edge distribution, the texture comparison, and the spatial correlation exceed the respective threshold difference value;
detecting a pose of the damaged vehicle in the image; and
determining which external vehicle parts are damaged in the image; and
calculating an estimated repair cost for the damaged vehicle based on which external vehicle parts are damaged based on accessing a parts database that includes repair costs.

### Claim 18 (depends on 17)

The computing device of claim 17, wherein the detecting the pose of the damaged vehicle in the image comprises:
training a first Convolutional Neural Networks (CNN) of a plurality of CNNs to detect the pose of a damaged vehicle in the image; and
training each of the plurality of CNNs to detect damage on a respective vehicle part of a plurality of external vehicle parts; and
executing the first CNN to detect the pose of the damaged vehicle in the image.

### Claim 19 (depends on 17)

The computing device of claim 17, wherein detecting damage to the set of parts further includes inferring damage to internal parts of the damaged vehicle from the determined damaged external vehicle parts; and
wherein calculating the estimated repair cost for the damaged vehicle is further based on which internal vehicle parts are inferred to be damaged.

### Claim 20 (depends on 19)

The computing device of claim 19, wherein the inferring damage to internal parts of the vehicle from the determined damaged external vehicle parts comprises executing a Markov Random Field (MRF) algorithm.