# US 2022/0067233 A1 · Generating operational and realistic models of physical systems

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0067233 A1
- Document ID: US20220067233A1
- Status: Published application
- Kind: A1
- Application number: 17/459,608
- Filed: 2021-08-27
- Published: 2022-03-03
- Earliest claimed: 2020-08-27
- Assignees: Industrial Technology and Safety Council LLC
- Applicants: Industrial Technology and Safety Council LLC
- Inventors: John A. Blackwell, II; Joshua M. Chapman
- Web page: https://patentgrep.com/patent/US20220067233A1

## CPC

- G06F 30/13: Architectural design, e.g. computer-aided architectural design [CAAD] related to design of buildings, bridges, landscapes, production plants or roads
- G06F 30/18: Network design, e.g. design based on topological or interconnect aspects of utility systems, piping, heating ventilation air conditioning [HVAC] or cabling (circuit design at the physical level G06F30/39; network planning tools for wireless communication networks H04W16/18)
- A63F 13/213: comprising photodetecting means, e.g. cameras, photodiodes or infrared cells (A63F13/219 takes precedence)
- A63F 13/65: automatically by game devices or servers from real world data, e.g. measurement in live racing competition
- A63F 13/69: by enabling or updating specific game elements, e.g. unlocking hidden features, items, levels or versions
- G09B 15/023: {Electrically operated}
- G06F 2113/14: Pipes
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- G09B: EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS (devices for psychotechnics or for testing reaction times A61B5/16; games, sports, amusements A63; projectors, projector screens G03B)

## Abstract

An exemplary method of generating a model of a physical environment includes generating a physical model of the physical environment using measured data from the physical environment, where the physical model includes spatial data about objects in the physical environment, correlating the physical model of the physical environment with a process model including components in the physical environment and interconnections between the components reflecting connections between the components in the physical environment, and generating the model of the physical environment by correlating the physical model of the physical environment with components of the process model based on information from a relational model associated with the physical environment, where the relational model includes probability distributions regarding component attributes of the components and relationships between components.

## Claims

### Claim 1

A method of generating a model of a physical environment in a digital environment comprising:
generating a physical model of the physical environment using data collected from the physical environment, the physical model including spatial data about objects in the physical environment;
correlating the physical model of the physical environment with a process model including components in the physical environment and interconnections between the components reflecting connections between the components in the physical environment; and
generating the model of the physical environment by correlating the physical model of the physical environment with components of the process model based on information from a relational model associated with the physical environment, wherein the relational model comprises probability distributions regarding component attributes of the components and relationships between the components.

### Claim 2 (depends on 1)

The method of claim 1, wherein the physical model comprises color and depth data determined from a scan of the physical environment collecting spatial data about the objects within the physical environment.

### Claim 3 (depends on 2)

The method of claim 2, wherein the scan of the physical environment includes one or more of a LiDAR scan of the physical environment, a sonar scan of the physical environment, and an infrared (IR) scan of the physical environment.

### Claim 4 (depends on 1)

The method of claim 1, wherein the process model is derived from one or more of a piping and instrumentation diagram (P&ID), a two-dimensional CAD model, and a map of the physical environment.

### Claim 5 (depends on 1)

The method of claim 1, wherein correlating the physical model of the physical environment with the process model comprises traversing a graph of the physical model and a graph of the process model to match components of the physical model with the components of the process model.

### Claim 6 (depends on 1)

The method of claim 1, wherein the component attributes include one or more of pose, size, or shape of the components.

### Claim 7 (depends on 1)

The method of claim 1, further comprising:
generating a digital twin of the physical environment using the model and a model library including models corresponding to the components, wherein the models corresponding to the components include information allowing the digital twin to mimic real life functionality of the components.

### Claim 8

A system for generating a digital twin of a physical environment comprising:
a relational model comprising probability distributions regarding attributes of components in the physical environment and relationships between the components within the physical environment;
a semantic model generated based on a correlation between a physical model of the physical environment including spatial data about objects within the physical environment and a process model including the components in the physical environment and interconnections between the components reflecting connections between the components in the physical environment, wherein the correlation between the physical model and the process model is based on the relational model; and
a model library including parametric models of the components, wherein the digital twin is generated by using the semantic model and the parametric models of the components.

### Claim 9 (depends on 8)

The system of claim 8, wherein the process model is generated using one or more of a piping and instrumentation diagram of the physical environment, a description of the physical environment, a map of the physical environment, and a schematic of the physical environment.

### Claim 10 (depends on 8)

The system of claim 8, wherein the physical model comprises color and depth data derived from a scan of the physical environment collecting spatial data about objects within the physical environment.

### Claim 11 (depends on 10)

The system of claim 10, wherein the scan of the physical environment includes one or more of a LiDAR scan of the physical environment, a sonar scan of the physical environment, and an infrared (IR) scan of the physical environment.

### Claim 12 (depends on 8)

The system of claim 8, wherein the correlation between the physical model and the process model is generated by traversing a graph of the physical model and a graph of the process model to match the objects of the physical model with the components of the process model.

### Claim 13

One or more non-transitory computer readable media encoded with instructions which, when executed by one or more processors, cause the one or more processors to perform a process comprising:
generating a physical model of a physical environment using data collected from the physical environment, the physical model including spatial data about objects in the physical environment;
generating a semantic model of the physical environment by correlating the objects in the physical model of the physical environment with components which may be located in the physical environment based on information from a relational model associated with the physical environment, wherein the relational model comprises probability distributions regarding component attributes of the components and relationships between the components in the physical environment; and
generating a digital twin of the physical environment using the semantic model and a model library including models corresponding to the components, wherein the models corresponding to the components include information allowing the digital twin to reflect real-life characteristics of the components.

### Claim 14 (depends on 13)

The one or more non-transitory computer readable media of claim 13, wherein the physical model comprises color and depth data derived from a scan of the physical environment collecting spatial data about the objects within the physical environment.

### Claim 15 (depends on 13)

The one or more non-transitory computer readable media of claim 13, wherein the scan of the physical environment includes one or more of a LiDAR scan of the physical environment, a sonar scan of the physical environment, and an infrared (IR) scan of the physical environment.

### Claim 16 (depends on 13)

The one or more non-transitory computer readable media of claim 13, wherein the process further comprises generating a process model of the physical environment from one or more of a piping and instrumentation diagram (P&ID), a two-dimensional CAD model, and a map of the physical environment, wherein the process model includes components in the physical environment and interconnections between the components reflecting connections between the components in the physical environment.

### Claim 17 (depends on 16)

The one or more non-transitory computer readable media of claim 16, wherein generating the semantic model comprises correlating the objects physical model of the physical environment with the components of the process model the process model by traversing a graph of the physical model and a graph of the process model to match components of the physical model with the components of the process model.

### Claim 18 (depends on 13)

The one or more non-transitory computer readable media of claim 13, wherein the component attributes include one or more of pose, size, or shape of the components.

### Claim 19 (depends on 13)

The one or more non-transitory computer readable media of claim 13, wherein the models corresponding to the components further include information allowing the digital twin to mimic real-life functionality of the components.

### Claim 20 (depends on 13)

The one or more non-transitory computer readable media of claim 13, wherein the process further comprises generating the relational model.
