# US 10,319,094 B1 · Technology for capturing, transmitting, and analyzing images of objects

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,319,094 B1
- Document ID: US10319094B1
- Status: Granted
- Kind: B1
- Application number: 15/160,403
- Filed: 2016-05-20
- Granted: 2019-06-11
- Earliest claimed: 2016-05-20
- Assignees: Ccc Information Services Inc.
- Applicants: Ccc Information Services Inc.
- Inventors: Ke Chen; John L. Haller; Athinodoros S. Georghiades; Takeo Kanade
- Web page: https://patentgrep.com/patent/US10319094B1

## CPC

- G06N 3/08: Learning methods
- G06T 7/75: {involving models}
- G06F 3/0484: for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F 16/5838: {using colour}
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/09: Supervised learning
- G06Q 50/00: Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism (healthcare informatics G16H)
- G06T 7/32: using correlation-based methods
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G06N 20/10: using kernel methods, e.g. support vector machines [SVM]
- G06T 2200/24: involving graphical user interfaces [GUIs]
- G06T 2207/10004: Still image; Photographic image
- G06T 2207/20024: Filtering details
- G06T 2207/20076: Probabilistic image processing
- G06T 2207/20081: Training; Learning
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR

## Abstract

A system for analyzing images of objects such as vehicles. According to certain aspects, the system includes a user interface device configured to capture a set of images depicting a target vehicle, and transfer the set of images to a server that stores a set of base image models. The server analyzes the set of images using a base image model corresponding to the target vehicle, a set of correlational filters, and a set of convolutional neural networks (CNNs) to determine a set of changes to the target vehicle as depicted in the set of images. The server further transmits, to the user interface device, information indicative of the set of changes for a user to view or otherwise access.

## Claims

### Claim 1

A system for capturing and analyzing images of vehicles, comprising:
a user interface device comprising:
a user interface,
an image sensor,
a device communication module, and
a device processor interfacing with the user interface, the image sensor, and the device communication module, the device processor configured to execute a set of instructions to cause the processor to:
cause the image sensor to capture a set of images depicting a target vehicle, and
transmit the set of images via the device communication module;
and a server comprising:
a database storing (i) a set of base object models, (ii) a set of correlation filters, and (iii) a set of convolutional neural networks (CNNs),
a memory storing an image processing routine,
a communication module, and
a processor interfacing with the database, the memory, and the communication module, and configured to execute the image processing routine to cause the processor to:
receive, via the communication module, the set of images from the user interface device,
access, from the database, (i) a base object model of the set of base object models corresponding to the target vehicle, (ii) at least a portion of the set of correlation filters corresponding to the base object model, and (iii) at least a portion of the set of CNNs corresponding to the base object model, each CNN included in at least the portion of the set of CNNs that correspond to the base object model trained on images of a respective set of components of the base object model,
analyze the set of images using the base object model, the at least the portion of the set of correlation filters, and the at least the portion of the set of CNNs, to determine a set of changes to the target vehicle as depicted in the set of images, including:
align, by utilizing the at least the portion of the set of correlation filters, the base object model with an orientation of the target vehicle depicted in the set of images based on one or more landmarks detected in the set of images;
detect, by utilizing one or more CNNs that are included in the at least the portion of set of CNNs and that correspond to a respective component of the target vehicle, one or more changes to the respective component of the target vehicle, the respective component determined based upon the aligned base model and the depiction of the target vehicle in the set of images;
transmit, to the user interface device via the communication module, information indicative of the set of changes, the user interface device configured to present, in the user interface, the information indicative of the set of changes.

### Claim 2 (depends on 1)

The system of claim 1, wherein the database of the server further stores a set of changed image files associated with the set of base object models, and a set of information files, wherein the set of changed image files illustrate a set of changes to a set of base objects represented in the set of base object models, and wherein the set of information files include information associated with the set of changed image files.

### Claim 3 (depends on 2)

The system of claim 2, wherein the memory of the server further stores an image processing training routine, and wherein the processor is further configured to execute the image processing training routine to cause the processor to:
for each base object represented in the set of base object models, determine, based on at least a portion of the set of changed image files and at least a portion of the set of information files, a corresponding correlation filter and a corresponding convolutional neural network (CNN), and
cause the database to store the corresponding correlation filter and the corresponding CNN.

### Claim 4 (depends on 1)

The system of claim 1, wherein to analyze the set of images to determine the set of changes, the processor is configured to:
analyze the set of images using the base object model, at least the portion of the set of correlation filters, and at least the portion of the set of CNNs, to detect damage to the target vehicle and a quantification of the damage to the target vehicle.

### Claim 5 (depends on 1)

The system of claim 1, wherein the processor is configured to execute the image processing routine to further cause the processor to:
determine that the analysis of the set of images is indeterminate, and
transmit, to the user interface device via the communication module, an indication that the analysis of the set of images is indeterminate, the user interface device configured to present the indication in the user interface.

### Claim 6 (depends on 1)

The system of claim 1, wherein the processor is configured to execute the image processing routine to further cause the processor to:
generate a heat map depicting the set of changes to the target vehicle, and
transmit the heat map to the user interface device via the communication module, the user interface device configured to present the heat map in the user interface.

### Claim 7 (depends on 1)

The system of claim 1, wherein the processor is configured to execute the image processing routine to further cause the processor to:
analyze at least a portion of the set of images to identify (i) the target vehicle that is depicted in at least the portion of the set of images, and (ii) a set of information associated with the target vehicle,
determine, based at least in part on the set of information associated with the target vehicle, that the set of images does not meet a threshold criteria for further image analysis using the base image model; and
in response to determining that the set of images does not meet the threshold criteria:
generate a notification indicating that the set of images does not meet the threshold criteria, and
transmit the notification to the user interface device via the communication module, the user interface device configured to present the notification in the user interface.

### Claim 8 (depends on 7)

The system of claim 7, wherein the device processor is configured to execute the set of instructions to further cause the processor to:
cause the image sensor to capture an additional set of images depicting the target vehicle, and
transfer the additional set of images via the device communication module; and
wherein the processor is configured to execute the image processing routine to further cause the processor to:
receive, via the communication module, the additional set of images from the user interface device,
analyze at least a portion of the additional set of images to identify an additional set of information associated with the target vehicle, and
determine, based at least in part on the additional set of information associated with the target vehicle, that a combination of the set of images and the additional set of images meets the threshold criteria for further image analysis using the base image model.

### Claim 9 (depends on 7)

The system of claim 7, wherein the processor is configured to execute the image processing routine to further cause the processor to:
determine image information that is needed to meet the threshold criteria for further image analysis using the base image model,
wherein to generate the notification, the processor is configured to:
generate the notification indicating at least the image information that is needed to meet the threshold criteria for further image analysis using the base image model.

### Claim 10 (depends on 1)

The system of claim 1, wherein the processor is configured to execute the image processing routine to further cause the processor to:
performing a multi-dimensional alignment between a base object of the base object model and the target object depicted in the set of images.

### Claim 11

A system for analyzing images of vehicles, comprising:
a database storing (i) a set of base object models, (ii) a set of correlation filters, and (iii) a set of convolutional neural networks (CNNs);
a memory storing an image processing routine;
a communication module; and
a processor interfacing with the database, the memory, and the communication module, and configured to execute the image processing routine to cause the processor to:
receive, via the communication module, a set of images from a user interface device, the set of images depicting a target vehicle,
access, from the database, (i) a base object model of the set of base object models corresponding to the target vehicle, (ii) at least a portion of the set of correlation filters corresponding to the base object model, and (iii) at least a portion of the set of CNNs corresponding to the base object model, each CNN included in at least the portion of the set of CNNs that correspond to the base object model trained on images of a respective set of components of the base object model,
analyze the set of images using the base object model, at least the portion of the set of correlation filters, and at least the portion of the set of CNNs, to determine a set of changes to the target vehicle as depicted in the set of images, including:
align, by utilizing the at least the portion of the set of correlation filters, the base object model with an orientation of the target vehicle depicted in the set of images based on one or more landmarks detected in the set of images;
detect, by utilizing one or more CNNs that are included in the at least the portion of set of CNNs and that correspond to a respective component of the target vehicle, one or more changes to the respective component of the target vehicle, the respective component determined based upon the aligned base model and the depiction of the target vehicle in the set of images; and
transmit, to the user interface device via the communication module, information indicative of the set of changes, the user interface device configured to present, in a user interface, the information indicative of the set of changes.

### Claim 12 (depends on 11)

The system of claim 11, further comprising:
the user interface device comprising:
the user interface,
an image sensor,
a device communication module, and
a device processor interfacing with the user interface, the image sensor, and the device communication module, the device processor configured to execute a set of instructions to cause the processor to:
cause the image sensor to capture the set of images depicting the target vehicle, and
transmit the set of images via the device communication module for receipt by the communication module.

### Claim 13 (depends on 12)

The system of claim 12, wherein the processor is configured to execute the image processing routine to further cause the processor to:
determine that the analysis of the set of images is indeterminate, and
transmit, to the user interface device via the communication module, an indication that the analysis of the set of images is indeterminate, the user interface device configured to present the indication in the user interface.

### Claim 14 (depends on 12)

The system of claim 12, wherein the processor is configured to execute the image processing routine to further cause the processor to:
generate a heat map depicting the set of changes to the target vehicle, and
transmit the heat map to the user interface device via the communication module, the user interface device configured to present the heat map in the user interface.

### Claim 15 (depends on 12)

The system of claim 12, wherein the processor is configured to execute the image processing routine to further cause the processor to:
analyze at least a portion of the set of images to identify (i) the target vehicle that is depicted in at least the portion of the set of images, and (ii) a set of information associated with the target vehicle,
determine, based at least in part on the set of information associated with the target vehicle, that the set of images does not meet a threshold criteria for further image analysis using the base image model; and
in response to determining that the set of images does not meet the threshold criteria:
generate a notification indicating that the set of images does not meet the threshold criteria, and
transmit the notification to the user interface device via the communication module, the user interface device configured to present the notification in the user interface.

### Claim 16 (depends on 15)

The system of claim 15, wherein the device processor is configured to execute the set of instructions to further cause the processor to:
cause the image sensor to capture an additional set of images depicting the target vehicle, and
transmit the additional set of images via the device communication module; and
wherein the processor is configured to execute the image processing routine to further cause the processor to:
receive, via the communication module, the additional set of images from the user interface device,
analyze at least a portion of the additional set of images to identify an additional set of information associated with the target vehicle, and
determine, based at least in part on the additional set of information associated with the target vehicle, that a combination of the set of images and the additional set of images meets the threshold criteria for further image analysis using the base image model.

### Claim 17 (depends on 11)

The system of claim 11, wherein the database further stores a set of changed image files associated with the set of base object models, and a set of information files, wherein the set of changed image files illustrate a set of changes to a set of base objects represented in the set of base object models, and wherein the set of information files include information associated with the set of changed image files.

### Claim 18 (depends on 17)

The system of claim 17, wherein the memory further stores an image processing training routine, and wherein the processor is further configured to execute the image processing training routine to cause the processor to:
for each base object represented in the set of base object models, determine, based on at least a portion of the set of changed image files and at least a portion of the set of information files, a corresponding correlation filter and a corresponding convolutional neural network (CNN), and
cause the database to store the corresponding correlation filter and the corresponding CNN.

### Claim 19 (depends on 11)

The system of claim 11, wherein to analyze the set of images to determine the set of changes, the processor is configured to:
analyze the set of images using the base object model, at least the portion of the set of correlation filters, and at least the portion of the set of CNNs, to detect damage to the target vehicle and a quantification of the damage to the target vehicle.

### Claim 20 (depends on 11)

The system of claim 11, wherein the processor is configured to execute the image processing routine to further cause the processor to:
perform a multi-dimensional alignment between a base object of the base object model and the target object depicted in the set of images.