US 11,080,838 B1Granted

Systems and methods for image labeling using artificial intelligence

AssigneeState Farm Mutual Automobile Insurance Company

InventorsYuntao Li, Charlie Isaksson, Marigona Bokshi-Drotar, Jing Wan

Life of this patent

Estimating the term…

Loading patent…

Abstract

An image analysis (“IA”) computer system for analyzing images of hail damage includes at least one processor in communication with at least one memory device. The at least one processor is programmed to: (i) store a damage prediction model associated with a rooftop, wherein the damage prediction model utilizes an artificial intelligence algorithm; (ii) display, to a user, an image of a rooftop; (iii) receive, from the user, a request to analyze damage to the rooftop; (iv) apply, by the at least one processor, the damage prediction model to the image, the damage prediction model outputting a plurality of damage prediction locations of the rooftop in relation to the image; and/or (v) display, by the at least one processor, an overlay box at each of the plurality of damage prediction locations, the overlay box being a virtual object overlaid onto the image for labeling the damage prediction locations.

Claims20

Dependent claims

An image analysis (“IA”) computer system for analyzing images of hail damage using artificial intelligence, the IA computer system including at least one processor in communication with at least one memory device, the at least one processor programmed to:

store, in the at least one memory device, a damage prediction model associated with a rooftop, wherein the damage prediction model utilizes an artificial intelligence algorithm, including machine learning techniques for analyzing rooftops and identifying damage thereto;
display, to a user on a user interface of a user computing device, an image of a rooftop;
receive, from the user via the user interface of the user computing device, a request to analyze damage to the rooftop displayed in the image;
in response to the request, apply, by the at least one processor, the image to the damage prediction model, the damage prediction model outputting a plurality of damage prediction locations of the rooftop in relation to the image;
display, by the at least one processor, on the user interface of the user computing device, an overlay box at each of the plurality of damage prediction locations, the overlay box being a virtual object overlaid onto the image displayed on the user interface;
receive, from the user computing device, a user selection of a location on the image displayed on the user interface;
when the user-selected location corresponds to one overlay box at one of the plurality of damage prediction locations:
display additional information associated with the corresponding damage prediction location;
and
receive, from the user computing device, a user interaction verifying whether the corresponding damage prediction location correlates to an actual damage location;
and
when the user-selected location does not correspond to any overlay box:
add, in response to the user selection of the location, a new overlay box including a new virtual overlay at the user-selected location;
and
determine whether the new overlay box correlates to an actual damage location.

12 more elements

A computer-implemented method for analyzing images of hail damage using artificial intelligence, the method implemented using an image analysis (“IA”) computer system, the IA computer system including at least one processor in communication with at least one memory device, the method comprising:

storing, in the at least one memory device, a damage prediction model associated with a rooftop, wherein the damage prediction model utilizes an artificial intelligence algorithm, including machine learning techniques for analyzing rooftops and identifying damage thereto;
displaying, to a user on a user interface of a user computing device, an image of a rooftop;
receiving, from the user via the user interface of the user computing device, a request to analyze damage to the rooftop displayed in the image;
in response to the request, applying, by the at least one processor, the image to the damage prediction model, the damage prediction model outputting a plurality of damage prediction locations of the rooftop in relation to the image;
displaying, by the at least one processor, on the user interface of the user computing device, an overlay box at each of the plurality of damage prediction locations, the overlay box being a virtual object overlaid onto the image displayed on the user interface;
receiving, from the user computing device, a user selection of a location on the image displayed on the user interface;
when the user-selected location corresponds to one overlay box at one of the plurality of damage prediction locations:
displaying additional information associated with the corresponding damage prediction location;
and
receiving, from the user computing device, a user interaction verifying whether the corresponding damage prediction location correlates to an actual damage location;
and
when the user-selected location does not correspond to any overlay box:
adding, in response to the user selection of the location, a new overlay box including a new virtual overlay at the user-selected location;
and
determining whether the new overlay box correlates to an actual damage location.

12 more elements

At least one non-transitory computer-readable storage media having computer-executable instructions embodied thereon, wherein when executed by at least one processor, the computer-executable instructions cause the at least one processor to:

store a damage prediction model associated with a rooftop, wherein the damage prediction model utilizes an artificial intelligence algorithm, including machine learning techniques for analyzing rooftops and identifying damage thereto;
display, to a user on a user interface of a user computing device, an image of a rooftop;
receive, from the user via the user interface of the user computing device, a request to analyze damage to the rooftop displayed in the image;
in response to the request, apply, by the at least one processor, the image to the damage prediction model, the damage prediction model outputting a plurality of damage prediction locations of the rooftop in relation to the image;
display, by the at least one processor, on the user interface of the user computing device, an overlay box at each of the plurality of damage prediction locations, the overlay box being a virtual object overlaid onto the image displayed on the user interface;
receive, from the user computing device, a user selection of a location on the image displayed on the user interface;
when the user-selected location corresponds to one overlay box at one of the plurality of damage prediction locations:
display additional information associated with the corresponding damage prediction location;
and
receive, from the user computing device, a user interaction verifying whether the corresponding damage prediction location correlates to an actual damage location;
and
when the user-selected location does not correspond to any overlay box:
add, in response to the user selection of the location, a new overlay box including a new virtual overlay at the user-selected location;
and
determine whether the new overlay box correlates to an actual damage location.

12 more elements

Dependencies are inferred from the claim text.

Facts

Application: 16/535,322Filed 8 Aug 2019 · us provisional application
Priority: 13 Aug 2018
Granted: 3 Aug 2021
Term ends: Not estimatedThe source records don’t support an estimate.

Classification13 codes

G06T 7/0004
Inspection of images, e.g. flaw detection: industrial image inspection
G06T 7/0002
Image analysis: inspection of images, e.g. flaw detection
G06V 10/7784
Active pattern-learning, e.g. online learning of image or video features: based on feedback from supervisors
G06V 20/17
Terrestrial scenes: taken from planes or by drones