# US 11,113,768 B1 · Systems and methods for enhanced situation visualization

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,113,768 B1
- Document ID: US11113768B1
- Status: Granted
- Kind: B1
- Application number: 16/687,059
- Filed: 2019-11-18
- Granted: 2021-09-07
- Earliest claimed: 2015-06-26
- Assignees: State Farm Mutual Automobile Insurance Company
- Applicants: State Farm Mutual Automobile Insurance Company
- Inventors: Timothy Joel Davis; Amy Engelhorn
- Web page: https://patentgrep.com/patent/US11113768B1

## CPC

- G06Q 40/08: Insurance
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

An augmented reality (AR) system for generating and displaying an enhanced situation visualization (ESV) is provided. The AR system may include an ESV computing device, a user computing device operated by a user and a reference database. The user computing device may transmit a reference request message to the ESV device, the reference request message including an image representative of a current view of a user. The ESV computing device may determine a subject of the image, retrieve reference information associated with the subject from the reference database, and determine situation information specific to the subject. The reference and situation information may be displayed on the user computing device to provide an ESV of the subject. The ESV may be used for insurance-related activities, such as handling, adjusting, and/or generating homeowner's insurance claims, and/or for instructional guidance.

## Claims

### Claim 1

An enhanced situation visualization (ESV) computing device comprising a memory and at least one processor communicatively coupled to the memory, wherein the at least one processor is programmed to:
receive a reference request message from a user computing device, the reference request message including an image of a property in a current state;
retrieve, from a reference database, reference information that includes a historical image of the property in a prior state;
determine, based upon the reference request message and the reference information, situation information that includes at least one difference between the prior state and the current state of the property;
generate a virtual grid of the at least one difference between the current state and the prior state;
generate a reference response message including the situation information, the reference information, and the virtual grid; and
transmit the reference response message to the user computing device, wherein the reference response message causes the user computing device to display at least one of the situation information and the reference information as overlaid upon a current view of the user to provide an enhanced situation visualization of the property, including displaying the virtual grid over the current view of the property as a virtual grid overlay.

### Claim 2 (depends on 1)

The ESV computing device of claim 1, wherein the at least one processor is further programmed to:
identify an insurance policy associated with the property; and
transmit the reference response message to an insurance server associated with a provider of the insurance policy.

### Claim 3 (depends on 2)

The ESV computing device of claim 2, wherein the at least one processor is further programmed to:
receive a claims form from the insurance server;
populate at least one field of the claims form with at least one of the reference information and the situation information; and
transmit the populated claims form to the insurance server.

### Claim 4 (depends on 1)

The ESV computing device of claim 1, wherein the reference request message further includes location information, and wherein the at least one processor is further programmed to identify, using the location information, a location of the user computing device.

### Claim 5 (depends on 1)

The ESV computing device of claim 1, wherein the at least one processor is further programmed to:
identify a beacon associated with the property, wherein the beacon is coupled to an item of interest associated with the property;
transmit an interrogation signal to the beacon;
determine a location of the beacon relative to the user computing device; and
transmit the location of the beacon relative to the user computing device in the reference response message for display on the user computing device as an icon including an item tag, a direction indicator, and a distance indicator.

### Claim 6 (depends on 1)

The ESV computing device of claim 1, wherein the at least one processor is further programmed to:
determine an object associated with a public utility;
determine a current state of the object; and
transmit an indicator of the current state of the object in the reference response message for display on the user computing device as an icon.

### Claim 7

A computer-based method for generating and displaying an enhanced situation visualization (ESV) on a user interface of a user computing device, said method comprising:
receiving, by an ESV computing device, a reference request message from the user computing device, the reference request message including an image of a property in a current state;
retrieving, from a reference database, reference information that includes a historical image of the property in a prior state;
determining, based upon the reference request message and the reference information, situation information that includes at least one difference between the prior state and the current state of the property;
generating a virtual grid of the at least one difference between the current state and the prior state;
generating a reference response message including the situation information, the reference information, and the virtual grid; and
transmitting the reference response message to the user computing device, wherein the reference response message causes the user computing device to display at least one of the situation information and the reference information as overlaid upon a current view of the user to provide an enhanced situation visualization of the property, including displaying the virtual grid over the current view of the property as a virtual grid overlay.

### Claim 8 (depends on 7)

The computer-based method of claim 7 further comprising:
identifying an insurance policy associated with the property; and
transmitting the reference response message to an insurance server associated with a provider of the insurance policy.

### Claim 9 (depends on 8)

The computer-based method of claim 8 further comprising:
receiving a claims form from the insurance server;
populating at least one field of the claims form with at least one of the reference information and the situation information; and
transmitting the populated claims form to the insurance server.

### Claim 10 (depends on 7)

The computer-based method of claim 7, wherein the reference request message further includes location information.

### Claim 11 (depends on 7)

The computer-based method of claim 7 further comprising:
identifying a beacon associated with the property, wherein the beacon is coupled to an item of interest associated with the property;
transmitting an interrogation signal to the beacon;
determining a location of the beacon relative to the user computing device; and
transmitting the location of the beacon relative to the user computing device in the reference response message for display on the user computing device as an icon including an item tag, a direction indicator, and a distance indicator.

### Claim 12 (depends on 7)

The computer-based method of claim 7 further comprising:
determining an object associated with a public utility;
determining a current state of the object; and
transmitting an indicator of the current state of the object in the reference response message for display on the user computing device as an icon.

### Claim 13

One or more non-transitory computer-readable storage media having computer-executable instructions embodied thereon, wherein when executed by at least one processor, the computer-executable instructions cause the processor to:
receive a reference request message from a user computing device, the reference request message including an image of a property in a current state;
retrieve, from a reference database, reference information that includes a historical image of the property in a prior state;
determine, based upon the reference request message and the reference information, situation information that includes at least one difference between the prior state and the current state of the property;
generate a virtual grid of the at least one difference between the current state and the prior state;
generate a reference response message including the situation information, the reference information, and the virtual grid; and
transmit the reference response message to the user computing device, wherein the reference response message causes the user computing device to display at least one of the situation information and the reference information as overlaid upon a current view of the user to provide an enhanced situation visualization of the property, including displaying the virtual grid over the current view of the property as a virtual grid overlay.

### Claim 14 (depends on 13)

The computer-readable storage media of claim 13, wherein the computer-executable instructions further cause the processor to:
identify an insurance policy associated with the property; and
transmit the reference response message to an insurance server associated with a provider of the insurance policy.

### Claim 15 (depends on 14)

The computer-readable storage media of claim 14, wherein the computer-executable instructions further cause the processor to:
receive a claims form from the insurance server;
populate at least one field of the claims form with at least one of the reference information and the situation information; and
transmit the populated claims form to the insurance server.

### Claim 16 (depends on 13)

The computer-readable storage media of claim 13, wherein the reference request message further includes location information, and wherein the computer-executable instructions further cause the processor to identify, using the location information, a location of the user computing device.

### Claim 17 (depends on 13)

The computer-readable storage media of claim 13, wherein the computer-executable instructions further cause the processor to:
identify a beacon associated with the property, wherein the beacon is coupled to an item of interest associated with the property;
transmit an interrogation signal to the beacon;
determine a location of the beacon relative to the user computing device; and
transmit the location of the beacon relative to the user computing device in the reference response message for display on the user computing device as an icon including an item tag, a direction indicator, and a distance indicator.