# US 10,121,241 B1 · System and method for assessing a building structure > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,121,241 B1 - Document ID: US10121241B1 - Status: Granted - Kind: B1 - Application number: 15/295,530 - Filed: 2016-10-17 - Granted: 2018-11-06 - Earliest claimed: 2015-10-22 - Assignees: State Farm Mutual Automobile Insurance Company - Applicants: State Farm Mutual Automobile Insurance Company - Inventors: Michael Shawn Jacob - Web page: https://patentgrep.com/patent/US10121241B1 ## CPC - G06T 7/001: {using an image reference approach} - G06F 18/24: Classification techniques - G06T 7/90: Determination of colour characteristics - G06T 11/60: Creating or editing images; Combining images with text - G06V 10/764: using classification, e.g. of video objects - G06V 20/17: taken from planes or by drones - G06V 20/176: {Urban or other man-made structures} - G06T 2207/10024: Color image - G06T 2207/10032: Satellite or aerial image; Remote sensing - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A computer-implemented method for assessing a building structure. The method includes receiving information indicative of captured data, the captured data comprising an image of the building structure. The method includes receiving information indicative of an analysis of the captured data. The method includes based on the analysis of the captured data, performing an assessment of the one or more examined regions. Performing the assessment includes determining one or more classifications each corresponding to a different one of the one or more examined regions, and determining one or more indicators each corresponding to a different one of the one or more classifications. The method includes determining a mapping of the one or more indicators to the one or more examined regions. The method includes providing for display the one or more indicators overlaid onto the image according to the determined mapping. ## Claims ### Claim 1 A computer-implemented method for assessing a building structure, the method executed by one or more processors programmed to perform the method, the method comprising: receiving, by the one or more processors, information indicative of captured data, the captured data comprising a first image and a second image of the building structure, the second image captured at an earlier point in time than the first image, and the first image and the second image comprising respective sets of pixels; analyzing, by the one or more processors, each examined region of a plurality of examined regions of the first image included in the captured data, the each examined region depicting a respective portion of the building structure, and analyzing the each examined region of the first image including determining, for the each examined region of the first image, a respective percentage difference of outlier pixels between the each examined region of the first image and a corresponding region of the second image, the corresponding region of the second image depicting the respective portion of the building structure of the first image at the earlier point in time; assessing, by the one or more processors, the each examined region of the first image, including: (i) determining, based on the analysis of the captured data, a respective classification of the each examined region of the first image based on the respective percentage difference of outlier pixels, the respective classification being a respective grade or score indicative of a respective immediacy of need of repair of the respective portion of the building structure; and (ii) determining a respective indicator corresponding to the respective classification of the each examined region; determining, by the one or more processors, a mapping of the respective indicator corresponding to the each examined region of the first image to the respective portion of the building structure; and providing for display, by the one or more processors, the respective indicators of the plurality of examined regions overlaid onto the respective portions of the building structure depicted in the first image according to the determined mappings, thereby indicating relative priorities, across the plurality of examined regions, of respective replacements of respective one or more building segments depicted in the each examined region. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, further comprising: determining a respective percentage of outlier pixels of the each examined region of the first image based on a difference between a respective average pixel value of the each examined region of the first image and a threshold amount; and determining a respective percentage of outlier pixels of the corresponding region of the second image based on a difference between a respective average pixel value of the corresponding region of the second image and the threshold amount, and wherein determining the respective percentage difference of outlier pixels between the each examined region of the first image and the corresponding region of the second image is based upon the respective percentage of outlier pixels of the each examined region of the first image and the respective percentage of outlier pixels of the corresponding region of the second image. ### Claim 3 (depends on 1) The computer-implemented method of claim 1, wherein the respective indicators of the plurality of examined regions comprise one or more of a graphical image, text, or a digital effect. ### Claim 4 (depends on 3) The computer-implemented method of claim 3, wherein the graphical image comprises a grid. ### Claim 5 (depends on 1) The computer-implemented method of claim 1, wherein the respective indicators of the plurality of examined regions vary in one or more of color or luminous intensity according to the respective classifications of the plurality of examined regions. ### Claim 6 (depends on 1) The computer-implemented method of claim 1, wherein the each examined region of the first image depicts a respective segment of the building structure, and wherein analyzing the each examined region of the first image further comprises, for the each examined region of the first image: comparing the respective segment of the building structure depicted in the each examined region of the first image to the respective segment of the building structure depicted in the corresponding region of the second image at the earlier point in time; and based on the comparison, determining an evaluation of the respective segment of the building structure. ### Claim 7 (depends on 6) The computer-implemented method of claim 6, further comprising: receiving information indicative of a user input associated with a given respective indicator of the respective indicators of the plurality of examined regions; based on the given respective indicator and the analysis of plurality of examined regions, obtaining information indicative of the evaluation of the respective segment of the building structure according to the information indicative of the user input; and providing an output signal comprising the information indicative of the evaluation of the respective segment of the building structure. ### Claim 8 A system for assessing a building structure, the system comprising: a communication network; one or more processors communicatively coupled to the communication network; and a non-transitory computer-readable memory coupled to the one or more processors and storing thereon instructions that, when executed by the one or more processors, cause the system to: receive information indicative of captured data, the captured data comprising a first image and a second image of the building structure, the second image captured at an earlier point in time than the first image, and the first image and the second image comprising respective sets of pixels; analyze each examined region of a plurality of examined regions of the first image included in the captured data, the each examined region depicting a respective portion of the building structure, and the analysis of the each examined region of the first image including a determination, for the each examined region of the first image, of a respective percentage difference of outlier pixels between the each examined region of the first image and a corresponding region of the second image, the corresponding region of the second image depicting the respective portion of the building structure of the first image at the earlier point in time; assess the each examined region of the first image, the assessment of the each examined region including: (i) a determination, based on the analysis of the captured data, of a respective classification of the each examined region of the first image based on the respective percentage difference of outlier pixels, the respective classification being a respective grade or score indicative of a respective immediacy of need of repair of the respective portion of the building structure; and (ii) a determination of a respective indicator corresponding to the respective classification of the each examined region; determine a mapping of the respective indicator corresponding to the each examined region of the first image to the respective portion of the building structure; and provide for display the respective indicators of the plurality of examined regions overlaid onto the respective portions of the building structure depicted in the first image according to the determined mappings, thereby indicating relative priorities, across the plurality of examined regions, of respective replacements of respective one or more building segments depicted in the each examined region. ### Claim 9 (depends on 8) The system of claim 8, wherein the instructions stored on the non-transitory computer-readable memory, when executed by the one or more processors, cause the system further to: determine a respective percentage of outlier pixels of the each examined region of the first image based on a difference between a respective average pixel value of the each examined region of the first image and a threshold amount; and determine a respective percentage of outlier pixels of the corresponding region of the second image based on a difference between a respective average pixel value of the corresponding region of the second image and the threshold amount, and wherein the determination of the respective percentage difference of outlier pixels between the each examined region of the first image and the corresponding region of the second image is based upon the respective percentage of outlier pixels of the each examined region of the first image and the respective percentage of outlier pixels of the corresponding region of the second image. ### Claim 10 (depends on 8) The system of claim 8, wherein the respective indicators of the plurality of examined regions comprise one or more of a graphical image, text, or a digital effect. ### Claim 11 (depends on 10) The system of claim 10, wherein the graphical image comprises a grid. ### Claim 12 (depends on 8) The system of claim 8, wherein the respective indicators of the plurality of examined regions vary in one or more of color or luminous intensity according to the respective classifications of the plurality of examined regions. ### Claim 13 (depends on 8) The system of claim 8, wherein: the each examined region of the first image depicts a respective segment of the building structure; and the analysis of the each examined region of the first image further comprises, for the each examined region of the first image: a comparison of the respective segment of the building structure depicted in the each examined region of the first image to the respective segment of the building structure depicted in the corresponding region of the second image; and based on the comparison, determining an evaluation of the respective segment of the building structure. ### Claim 14 (depends on 13) The system of claim 13, wherein the instructions stored on the non-transitory computer-readable memory, when executed by the one or more processors, cause the system further to: receive information indicative of a user input associated with a given respective indicator of the respective indicators of the plurality of examined regions; based on the given respective indicator and the analysis of the plurality of examined regions, obtain information indicative of the evaluation of the respective segment of the building structure according to the information indicative of the user input; and provide an output signal comprising the information indicative of the evaluation of the respective segment of the building structure. ### Claim 15 A non-transitory, computer-readable medium storing instructions that when executed by one or more processors of a computer system, cause the computer system to: receive, via a communication network, information indicative of captured data, the captured data comprising a first image and a second image of the building structure, the second image captured at an earlier point in time than the first image, and the first image and the second image comprising respective sets of pixels; analyze each examined region of a plurality of examined regions of the first image included in the captured data, the each examined region depicting a respective portion of the building structure, and the analysis of the each examined region of the first image including a determination, for the each examined region of the first image, a respective percentage difference of outlier pixels between the each examined region of the first image and a corresponding region of the second image, the corresponding region of the second image depicting the respective portion of the building structure of the first image at the earlier point in time; assess the each examined region of the first image, the assessment of the each examined region including: (i) a determination, based on the analysis of the captured data, a respective classification of the each examined region of the first image based on the respective percentage difference of outlier pixels, the respective classification being a respective grade or score indicative of a respective immediacy of need for repair of the respective portion of the building structure; and (ii) a determination of a respective indicator corresponding to the respective classification of the each examined region; determine a mapping of the respective indicator corresponding to the each examined region of the first image to the respective portion of the building structure; and provide for display the respective indicators of the plurality of examined regions overlaid onto the respective portions of the building structure depicted in the first image according to the determined mappings, thereby indicating relative priorities, across the plurality of examined regions, of respective replacements of respective one or more building segments depicted in the each examined region. ### Claim 16 (depends on 15) The computer-readable medium of claim 15, wherein respective indicators of the plurality of examined regions comprise one or more of a graphical image, text, or a digital effect. ### Claim 17 (depends on 16) The computer-readable medium of claim 16, wherein the graphical image comprises a grid. ### Claim 18 (depends on 15) The computer-readable medium of claim 15, wherein the respective indicators of the plurality of examined regions vary in one or more of color or luminous intensity according to respective classifications of the plurality of examined regions. ### Claim 19 (depends on 15) The computer-readable medium of claim 15, wherein: the each examined region of the first image depicts a respective segment of the building structure; and the analysis of the each examined region of the first image further comprises, for the each examined region of the first image: a comparison of the respective segment of the building structure depicted in the each examined region of the first image to be respective segment of the building structure depicted in the corresponding region of the second image; and based on the comparison, determining an evaluation of the respective segment of the building structure. ### Claim 20 (depends on 19) The computer-readable medium of claim 19, wherein the instructions stored on the non-transitory computer-readable memory, when executed by the one or more processors, cause the system further to: receive information indicative of a user input associated with a given respective indicator of the respective indicators of the plurality of examined regions; based on the given respective indicator and the analysis of the plurality of examined regions, obtain information indicative of the evaluation of the respective segment of the building structure according to the information indicative of the user input; and provide an output signal comprising the information indicative of the evaluation of the respective segment of the building structure.