US 2026/0219639 A1Published application

Generative ai industrial automation event based prompt engineering

AssigneeRockwell Automation Technologies, Inc.

InventorsStephen C. Briant, Sharon Billi-Duran, Michael B. Miller, Maurizio Fumagalli, Steven J. Kowal

Life of this patent

Estimating the term…

Loading patent…

Abstract

A human-machine interface (HMI) development system leverages a generative AI model to assist in development of HMI projects in accordance with specified functional requirements, which can be provided to the development system as intuitive natural language spoken or written text. The system can formulate and implement HMI project edits during design time based on analysis of this natural language design input. After the HMI project is deployed as a runtime application, the system can also receive and process natural language requests to modify the runtime HMI in accordance with described modification criteria.

Claims20

Dependent claims

A system, comprising:

a memory that stores executable components;
and
a processor, operatively coupled to the memory, that executes the executable components, the executable components comprising:
a human-machine interface (HMI) deployment component configured to deploy an HMI application to an HMI terminal for execution, wherein the HMI application renders, on the HMI terminal, display screens comprising graphical objects that render data generated by an industrial automation system;
a user interface component configured to receive, via interaction with the HMI application, a natural language input that describes an event-driven action to be performed by the HMI application;
a generative artificial intelligence (AI) component configured to, in response to receipt of the natural language input, formulate a prompt, directed to a generative AI model, designed to obtain a response from the generative AI model comprising information used by the generative AI component to determine the event-driven action based on the natural language input and to formulate an edit that configures the HMI application to perform the event-driven action, wherein the prompt is generated based on analysis of the natural language input and one or more custom models trained with training data;
and
an HMI generation component configured to apply the edit to the HMI application.

5 more elements

A method, comprising:

deploying, by a system comprising a processor, a human-machine interface (HMI) application to an HMI terminal for execution, wherein the HMI application renders, on the HMI terminal, display screens comprising graphical objects that render data generated by an industrial automation system;
receiving, by the system via interaction with the HMI application, a natural language input that describes an event-driven action to be performed by the HMI application;
in response to the receiving of the natural language input, formulating, by the system, a prompt, directed to a generative artificial intelligence (AI) model, designed to obtain a response from the generative AI model comprising information used by the system to determine the event-driven action, wherein the prompt is generated based on analysis of the natural language input and one or more custom models trained with training data;
formulating, by the system, an edit that configures the HMI application to perform the event-driven action, wherein the formulating comprises formulating the edit based on the natural language input, the one or more custom models, and the response from the generative AI model;
and
applying, by the system, the edit to the HMI application.

3 more elements

A non-transitory computer-readable medium having stored thereon instructions that, in response to execution, cause a system comprising a processor to perform operations, the operations comprising:

deploying a human-machine interface (HMI) application to an HMI terminal for execution, wherein the HMI application renders, on the HMI terminal, display screens comprising graphical objects that render data generated by an industrial automation system;
receiving, via interaction with the HMI application, a natural language input that describes an event-driven action to be performed by the HMI application;
in response to the receiving of the natural language input, formulating, a prompt, directed to a generative artificial intelligence (AI) model, designed to obtain a response from the generative AI model comprising information used by the system to determine the event-driven action, wherein the prompt is generated based on analysis of the natural language input and one or more custom models trained with training data;
configuring the HMI application to perform the event-driven action based on the natural language input, the one or more custom models, and the response from the generative AI mode.

1 more element

Dependencies are inferred from the claim text.