# US 2026/0099141 A1 · Generating industrial process flow diagrams using generative ai and image recognition

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0099141 A1
- Document ID: US20260099141A1
- Status: Published application
- Kind: A1
- Application number: 18/907,102
- Filed: 2024-10-04
- Published: 2026-04-09
- Earliest claimed: 2024-10-04
- Assignees: Honeywell International Inc.
- Applicants: Honeywell International Inc.
- Inventors: Sandhya Beejady; Rajapriyan Thambidurai; Sridhar S; Dhandapani Krishnasamy
- Web page: https://patentgrep.com/patent/US20260099141A1

## CPC

- G05B 23/0286: {Modifications to the monitored process, e.g. stopping operation or adapting control}
- G05B 23/0216: {Human interface functionality, e.g. monitoring system providing help to the user in the selection of tests or in its configuration}
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)

## Abstract

Systems and methods for generating industrial process flow diagrams using generative AI and image recognition are described herein. In certain embodiments, a system includes memory devices that stores a cognitive services model trained to determine whether image data represents process information; and a generative model trained to generate process diagram information from the image data. The system also includes processors that receive the image data; and execute the cognitive services model to determine whether the image data contains information associated with a process. When the cognitive services model determines that the image data contains the information associated with the process, the processors are also execute the generative model using the image data to generate the process diagram information. Further, the processors provide the process diagram information to a diagram visualization program, wherein the diagram visualization program generates a diagram from the process diagram information.

## Claims

### Claim 1

A system comprising:
one or more memory devices configured to store:
a cognitive services model trained to determine whether image data represents process information; and
a generative model trained to generate process diagram information from the image data; and
one or more processors configured to:
receive the image data;
execute the cognitive services model to determine whether the image data contains information associated with a process;
when the cognitive services model determines that the image data contains the information associated with the process, execute the generative model using the image data to generate the process diagram information; and
provide the process diagram information to a diagram visualization program, wherein the diagram visualization program generates a diagram from the process diagram information.

### Claim 2 (depends on 1)

The system of claim 1, wherein, when generating the process diagram information, the generative model is trained to identify:
symbols depicted in the image data;
connections between the symbols; and
relationships implied between the connections.

### Claim 3 (depends on 2)

The system of claim 2, wherein the symbols are derived from a set of symbols defined within an industry standard.

### Claim 4 (depends on 3)

The system of claim 3, wherein the generative model is periodically trained to incorporate new symbols added to the industry standard.

### Claim 5 (depends on 1)

The system of claim 1, wherein the generative model is trained on at least one of:
a domain-specific subset of symbols within a set of symbols; and
a global set of symbols within the set of symbols representing multiple industry domains.

### Claim 6 (depends on 1)

The system of claim 1, wherein the image data contain at least one of:
image files;
configuration data;
asset information; and
inventory information.

### Claim 7 (depends on 1)

The system of claim 1, wherein the one or more processors is further configured to:
receive corrections made by users to the diagram generated from the process diagram information; and
perform additional training on at least one of the cognitive services model and the generative model based on the corrections.

### Claim 8 (depends on 1)

The system of claim 1, wherein the one or more processors is further configured to:
identify one or more definitions for approval in the image data;
receive updated annotations and updated components for the one or more definitions;
receive expected output when the updated components are provided as inputs to the generative model; and
validate the generative model using the updated components and the expected output.

### Claim 9 (depends on 1)

The system of claim 1, wherein the generative model generates image description data as at least one of javascript object notation information and hypertext markup language information, wherein a process flow diagram service renders diagrams based on the image description data.

### Claim 10 (depends on 1)

The system of claim 1, wherein at least one of the cognitive services model and the generative model are deployed to users through a cloud platform.

### Claim 11

A method comprising:
receiving image data;
determining whether the image data contains information associated with a process;
when the image data contains the information associated with the process, providing the image data as an input to a generative model, wherein the generative model generates process diagram information from the input; and
providing the process diagram information to a collaboration service, wherein the collaboration service generates a diagram from the process diagram information.

### Claim 12 (depends on 11)

The method of claim 11, further comprising training the generative model to identify:
symbols depicted in the image data;
connections between the symbols; and
relationships implied between the connections.

### Claim 13 (depends on 12)

The method of claim 12, wherein the symbols are derived from a set of symbols defined within an industry standard.

### Claim 14 (depends on 13)

The method of claim 13, wherein the generative model is periodically trained to incorporate new symbols added to the industry standard.

### Claim 15 (depends on 11)

The method of claim 11, wherein the generative model is trained on at least one of:
a domain-specific subset of symbols within a set of symbols; and
a global set of symbols within the set of symbols representing multiple industry domains.

### Claim 16 (depends on 11)

The method of claim 11, wherein the image data contain at least one of:
image files;
configuration data;
asset information; and
inventory information.

### Claim 17 (depends on 11)

The method of claim 11, further comprising:
receiving corrections made by users to the diagram generated from the process diagram information; and
performing additional training of the generative model based on the corrections.

### Claim 18 (depends on 11)

The method of claim 11, further comprising:
identifying one or more definitions for approval in the image data;
receiving updated annotations and updated components for the one or more definitions;
receiving expected output when the updated components are provided as inputs to the generative model; and
validating the generative model using the updated components and the expected output.

### Claim 19 (depends on 11)

The method of claim 11, wherein the process diagram information comprises at least one of javascript object notation information and hypertext markup language information, wherein a process flow diagram service renders diagrams based on the process diagram information.

### Claim 20

A system comprising:
one or more memory devices configured to store:
a cognitive services model trained to determine whether image data represents process information; and
a generative model trained to generate process diagram information from the image data; and
one or more processors configured to:
receive the image data;
execute the cognitive services model to determine whether the image data contains information associated with a process;
when the cognitive services model determines that the image data contains the information associated with the process, execute the generative model using the image data to generate the process diagram information;
provide the process diagram information to a diagram visualization program, wherein the diagram visualization program generates a diagram from the process diagram information;
receive updated diagram information for the diagram; and
validate the generative model using the updated diagram information.
