# US 10,873,724 B1 · Virtual environment generation for collaborative building assessment

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,873,724 B1
- Document ID: US10873724B1
- Status: Granted
- Kind: B1
- Application number: 16/242,206
- Filed: 2019-01-08
- Granted: 2020-12-22
- Earliest claimed: 2019-01-08
- Assignees: State Farm Mutual Automobile Insurance Company
- Applicants: State Farm Mutual Automobile Insurance Company
- Inventors: Rebecca A. Little; Bryan R. Nussbaum; Deanna L. Stockweather; Nathan C. Summers; An Ho; Vicki King
- Web page: https://patentgrep.com/patent/US10873724B1

## CPC

- H04N 7/157: {defining a virtual conference space and using avatars or agents (computer conference optimisation or adaptation H04L12/1827)}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 16/29: Geographical information databases
- G06Q 50/163: Real estate management
- G06T 19/003: {Navigation within 3D models or images}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Systems and methods for facilitating virtual collaboration between users within virtual environments are disclosed. A plurality of users access a collaborative virtual environment system using interface devices to establish a virtual collaboration session in which users communicate via a bidirectional real-time communication channel and interact within a virtual environment. Annotations or other user interactions within the virtual environment are automatically propagated to each user in the virtual collaboration session in order to facilitate collaboration. Data from such virtual collaboration sessions may be captured for later use.

## Claims

### Claim 1

A computer-implemented method for virtual collaboration between a plurality of users, comprising:
establishing, via a communication network, a first communication connection with a first virtual environment interface device associated with a first user of the plurality of users;
establishing, via the communication network, a second communication connection with a second virtual environment interface device associated with a second user of the plurality of users;
initiating, by one or more processors, a virtual collaboration session with the first user and the second user by (i) establishing a communication channel between the first user and the second user via the first communication connection and the second communication connection for bidirectional communication, (ii) providing a virtual environment representing a physical location including a building to the first virtual environment interface device and the second virtual environment interface device for presentation to the first user and the second user, and (iii) causing a plurality of data tools to be presented to at least the first user within the virtual environment, wherein the plurality of data tools presented to the first user include a view synchronization tool;
routing, via the communication channel, real-time communication data between the first user and the second user during the virtual collaboration session;
receiving, at the one or more processors, user interaction data from the first user, wherein the user interaction data indicates a portion of the building;
generating, by the one or more processors, a virtual overlay within the virtual environment based upon the user interaction data, wherein the virtual overlay is associated with a location within the virtual environment corresponding to the portion of the building;
updating, by the one or more processors, the virtual environment to include the virtual overlay;
synchronizing, by the one or more processors, the virtual environment presented to the first user and the second user, including causing the virtual overlay to be presented to the second user via the second virtual environment interface device;
receiving, by the one or more processors and from the first virtual environment interface device, an input indicating a selection of the view synchronization tool; and
causing, by the one or more processors and based at least in part on the input, a location and a viewing direction of a viewing perspective of the second user within the virtual environment to match a current location and a current viewing direction, respectively, of a current viewing perspective of the first user within the virtual environment.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, wherein the communication channel is a voice communication channel.

### Claim 3 (depends on 1)

The computer-implemented method of claim 1, wherein:
the user interaction data indicates a user annotation regarding an aspect of the portion of the building; and
the virtual overlay includes a visual representation of the user annotation.

### Claim 4 (depends on 1)

The computer-implemented method of claim 1, wherein:
the first user is a trainer,
the second user is a trainee, and
the view synchronization tool is one of multiple trainer-exclusive data tools available for use by the trainer and unavailable for use by the trainee.

### Claim 5 (depends on 1)

The computer-implemented method of claim 1, further comprising:
receiving, at the one or more processors, a plurality of images of the physical location including the building, wherein the plurality of images show the building from a plurality of viewing perspectives;
generating, by the one or more processors, a representation of the physical location including the building by photogrammetric analysis of the plurality of images; and
storing, in a computer memory, the representation of the physical location including the building,
wherein providing the virtual environment to the first virtual environment interface device and the second virtual environment interface device includes accessing the representation of the physical location from the computer memory and sending the representation of the physical location to the first virtual environment interface device and the second virtual environment interface device, respectively.

### Claim 6 (depends on 5)

The computer-implemented method of claim 5, further comprising:
capturing, by a mobile computing device associated with the second user, the plurality of images of the physical location including the building.

### Claim 7 (depends on 1)

The computer-implemented method of claim 1, further comprising:
receiving, at the one or more processors, additional user interaction data indicating a user selection of a data overlay within the virtual environment, wherein the data overlay is associated with the physical location, without reference to any portion of the physical location;
accessing, by the one or more processors, additional data associated with the user selection;
generating, by the one or more processors, an additional virtual overlay within the virtual environment based upon the additional data;
updating, by the one or more processors, the virtual environment to include the additional virtual overlay; and
synchronizing, by the one or more processors, the virtual environment presented to the first user and the second user, including causing the additional virtual overlay to be presented to the first user and the second user via the first virtual environment interface device and the second virtual environment interface device, respectively.

### Claim 8 (depends on 7)

The computer-implemented method of claim 7, wherein accessing the additional data includes retrieving one or more of the following types of external data from a database: property zoning data, property tax assessments, property environmental reports, property ownership records, or insurance policy records.

### Claim 9

A computer system for virtual collaboration between a plurality of users, comprising:
one or more processors;
a communication module communicatively connected to the one or more processors and configured to communicate with computing devices via a communication network;
a non-transitory program memory communicatively coupled to the one or more processors and storing executable instructions that, when executed by the one or more processors, cause the computer system to:
establish a first communication connection over the communication network with a first virtual environment interface device associated with a first user of the plurality of users via the communication module;
establish a second communication connection over the communication network with a second virtual environment interface device associated with a second user of the plurality of users via the communication module;
initiate a virtual collaboration session with the first user and the second user by (i) establishing a communication channel between the first user and the second user via the first communication connection and the second communication connection for bidirectional communication, (ii) providing a virtual environment representing a physical location including a building to the first virtual environment interface device and the second virtual environment interface device for presentation to the first user and the second user, and (iii) causing a plurality of data tools to be presented to at least the first user within the virtual environment, wherein the plurality of data tools presented to the first user include a view synchronization tool;
route real-time communication data between the first user and the second user during the virtual collaboration session via the communication channel;
receive user interaction data from the first user, wherein the user interaction data indicates a portion of the building;
generate a virtual overlay within the virtual environment based upon the user interaction data, wherein the virtual overlay is associated with a location within the virtual environment corresponding to the portion of the building;
update the virtual environment to include the virtual overlay;
synchronize the virtual environment presented to the first user and the second user, including causing the virtual overlay to be presented to the second user via the second virtual environment interface device;
receive from the first virtual environment interface device, an input indicating a selection of the view synchronization tool; and
cause, by the one or more processors and based at least in part on the input, a location and a viewing direction of a viewing perspective of the second user within the virtual environment to match a current location and a current viewing direction, respectively, of a current viewing perspective of the first user within the virtual environment.

### Claim 10 (depends on 9)

The computer system of claim 9, wherein the communication channel is a voice communication channel.

### Claim 11 (depends on 9)

The computer system of claim 9, wherein:
the user interaction data indicates a user annotation regarding an aspect of the portion of the building; and
the virtual overlay includes a visual representation of the user annotation.

### Claim 12 (depends on 9)

The computer system of claim 9, wherein:
the first user is a trainer,
the second user is a trainee, and
the view synchronization tool is one of multiple trainer-exclusive data tools available for use by the trainer and unavailable for use by the trainee.

### Claim 13 (depends on 9)

The computer system of claim 9, wherein:
the executable instructions further cause the computer system to:
receive a plurality of images of the physical location including the building, wherein the plurality of images show the building from a plurality of viewing perspectives;
generate a representation of the physical location including the building by photogrammetric analysis of the plurality of images; and
store the representation of the physical location including the building in the non-transitory program memory; and
the executable instructions that cause the computer system to provide the virtual environment to the first virtual environment interface device and the second virtual environment interface device cause the computer system to access the representation of the physical location from the non-transitory program memory and send the representation of the physical location to the first virtual environment interface device and the second virtual environment interface device, respectively.

### Claim 14 (depends on 9)

The computer system of claim 9, wherein the executable instructions further cause the computer system to:
receive additional user interaction data indicating a user selection of a data overlay within the virtual environment, wherein the data overlay is associated with the physical location, without reference to any portion of the physical location;
access additional data associated with the user selection;
generate an additional virtual overlay within the virtual environment based upon the additional data;
update the virtual environment to include the additional virtual overlay; and
synchronize the virtual environment presented to the first user and the second user, including causing the additional virtual overlay to be presented to the first user and the second user via the first virtual interface environment device and the second virtual environment interface device, respectively.

### Claim 15

A tangible, non-transitory computer-readable medium storing executable instructions for virtual collaboration between a plurality of users that, when executed by at least one processor of a computer system, cause the computer system to:
establish a first communication connection over a communication network with a first virtual environment interface device associated with a first user of the plurality of users;
establish a second communication connection over a communication network with a second virtual environment interface device associated with a second user of the plurality of users;
initiate a virtual collaboration session with the first user and the second user by (i) establishing a communication channel between the first user and the second user via the first communication connection and the second communication connection for bidirectional communication, (ii) providing a virtual environment representing a physical location including a building to the first virtual environment interface device and the second virtual environment interface device for presentation to the first user and the second user, and (iii) causing a plurality of data tools to be presented to at least the first user within the virtual environment, wherein the plurality of data tools presented to the first user include a view synchronization tool;
route real-time communication data between the first user and the second user during the virtual collaboration session via the communication channel;
receive user interaction data from the first user, wherein the user interaction data indicates a portion of the building;
generate a virtual overlay within the virtual environment based upon the user interaction data, wherein the virtual overlay is associated with a location within the virtual environment corresponding to the portion of the building;
update the virtual environment to include the virtual overlay;
synchronize the virtual environment presented to the first user and the second user, including causing the virtual overlay to be presented to the second user via the second virtual environment interface device;
receive from the first virtual environment interface device, an input indicating selection of the view synchronization tool; and
cause, based at least in part on the input, a location and a viewing direction of a viewing perspective of the second user within the virtual environment to match a current location and a current viewing direction, respectively, of a current viewing perspective of the first user within the virtual environment.

### Claim 16 (depends on 15)

The tangible, non-transitory computer-readable medium of claim 15, wherein the communication channel is a voice communication channel.

### Claim 17 (depends on 15)

The tangible, non-transitory computer-readable medium of claim 15, wherein:
the user interaction data indicates a user annotation regarding an aspect of the portion of the building; and
the virtual overlay includes a visual representation of the user annotation.

### Claim 18 (depends on 15)

The tangible, non-transitory computer-readable medium of claim 15, wherein:
the first user is a trainer,
the second user is a trainee, and
the view synchronization tool is one of multiple trainer-exclusive data tools available for use by the trainer and unavailable for use by the trainee.

### Claim 19 (depends on 15)

The tangible, non-transitory computer-readable medium of claim 15, wherein:
the executable instructions further cause the computer system to:
receive a plurality of images of the physical location including the building, wherein the plurality of images show the building from a plurality of viewing perspectives;
generate a representation of the physical location including the building by photogrammetric analysis of the plurality of images; and
store the representation of the physical location including the building in a computer memory; and
the executable instructions that cause the computer system to provide the virtual environment to the first virtual environment interface device and the second virtual environment interface device cause the computer system to access the representation of the physical location from the computer memory and send the representation of the physical location to the first virtual environment interface device and the second virtual environment interface device, respectively.

### Claim 20 (depends on 15)

The tangible, non-transitory computer-readable medium of claim 15, wherein the executable instructions further cause the computer system to:
receive additional user interaction data indicating a user selection of a data overlay within the virtual environment, wherein the data overlay is associated with the physical location, without reference to any portion of the physical location;
access additional data associated with the user selection;
generate an additional virtual overlay within the virtual environment based upon the additional data;
update the virtual environment to include the additional virtual overlay; and
synchronize the virtual environment presented to the first user and the second user, including causing the additional virtual overlay to be presented to the first user and the second user via the first virtual environment interface device and the second virtual environment interface device, respectively.