# US 12,216,454 B2 · Extension device for an automation device > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,216,454 B2 - Document ID: US12216454B2 - Status: Granted - Kind: B2 - Application number: 17/296,250 - Filed: 2019-10-28 - Published: 2022-01-20 - Granted: 2025-02-04 - Earliest claimed: 2018-11-23 - Assignees: Siemens Aktiengesellschaft - Applicants: Siemens Aktiengesellschaft - Inventors: Norman Drews; Johannes Frank; Andreas Macher; Josep Soler Garrido; Ingo Thon; Renè Fischer; Heiko Claussen - Web page: https://patentgrep.com/patent/US12216454B2 ## CPC - G05B 19/0421: {Multiprocessor system} - G05B 19/41835: {characterised by program execution} - G05B 19/05: Programmable logic controllers, e.g. simulating logic interconnections of signals according to ladder diagrams or function charts - G05B 2219/25255: Neural network - G05B 2219/31229: Supervisor, leader, workstation controller, automation, machine control - G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G) ## Abstract An extension device for one or more automation devices in an industrial system is provided. Industrial data processing units capable of performing data processing based on one or more artificial neural networks are provided. To enable and/or accelerate one or more computations in an industrial system, thereby simplifying integration of artificial intelligence into the industrial system, and to simplify data exchange between an extension device capable of processing data using artificial intelligence and an automation device, one or more results of the one or more computations are obtained. The results indicate one or more states of the industrial system. The one or more results are provided via a process state model shared with the automation device to monitor and/or control the industrial system. ## Claims ### Claim 1 An extension device for an automation device in an industrial system, the extension device being operable to enable, accelerate, or enable and accelerate one or more computations, wherein the extension device is directly connected to a camera via a peripheral interface, the extension device comprising: a processor configured to: perform the one or more computations based at least in part on data collected from the camera by the extension device; obtain one or more results of the one or more computations, wherein the one or more results indicate one or more states of the industrial system, wherein an availability of a new result of the one or more results is indicated by a status bit in a process state model shared between the extension device and the automation device; and provide the one or more results via the process state model to monitor, control, or monitor and control the industrial system, wherein the process state model is a data structure and comprises state representations of devices of the industrial system, the state representations being based on data indicative of one or more characteristics of the devices of the industrial system, and wherein the extension device is operable to provide an indicator via the process state model that an image has been captured and is available for further processing. ### Claim 2 (depends on 1) The extension device of claim 1, wherein the extension device is a neural processing unit configured to compute one or more artificial neural networks. ### Claim 3 (depends on 1) The extension device of claim 1, wherein the processor is further configured to apply logic to an intermediate result of the one or more computations, to the one or more states, or to a combination thereof. ### Claim 4 (depends on 1) The extension device of claim 1, wherein the processor is further configured to update at least parts of the process state model with one or more of the results. ### Claim 5 (depends on 1) The extension device of claim 1, wherein the processor comprises a processing unit configured to perform the one or more computations. ### Claim 6 (depends on 1) The extension device of claim 1, wherein the processor comprises a processing unit configured to perform at least part of the one or more computations with a SIMD-architecture. ### Claim 7 (depends on 1) The extension device of claim 1, wherein the processor comprises a processing unit based on a neural network accelerating architecture. ### Claim 8 (depends on 1) The extension device of claim 1, wherein the processor comprises a processing unit configured to provide computations with 0,1 or more TOPS/Watt. ### Claim 9 (depends on 1) The extension device of claim 1, wherein the processor is operative to obtain industrial data via the process state model. ### Claim 10 (depends on 1) The extension device of claim 1, further comprising a control unit configured to retrieve, provide, or retrieve and provide data to or from the automation device via the process state model. ### Claim 11 (depends on 1) The extension device of claim 1, further comprising a communication interface operative to exchange at least parts of the process state model between the automation device and the extension device. ### Claim 12 (depends on 1) The extension device of claim 1, further comprising a peripheral connectivity system operative to provide connectivity to data sources connected to the extension device. ### Claim 13 (depends on 1) The extension device of claim 1, wherein the automation device is configured to trigger a command in the extension device by setting a flag in the process state model. ### Claim 14 (depends on 1) The extension device of claim 1, wherein the indicator is a state representation. ### Claim 15 (depends on 1) The extension device of claim 1, wherein the status bit is a binary state. ### Claim 16 A system comprising: one or more automation devices in an industrial system; and one or more extension devices, each extension device of the one or more extension devices being operable to enable, accelerate, or enable and accelerate one or more computations, wherein the respective extension device is directly connected to a camera via a peripheral interface, the respective extension device comprising: a processor configured to: perform the one or more computations based at least in part on data collected from the camera by the respective extension device; obtain one or more results of the one or more computations, wherein the one or more results indicate one or more states of the industrial system; and provide the one or more results via a process state model shared between the respective extension device and a respective automation device of the one or more automation devices to monitor, control, or monitor and control the industrial system, wherein the process state model is a data structure and comprises state representations of devices of the industrial system, the state representations being based on data indicative of one or more characteristics of the devices of the industrial system, wherein at least one of the one or more automation devices is operative to monitor, control, or monitor and control the industrial system according to at least one result of at least one computation performed by the one or more extension devices, wherein an availability of the at least one result is indicated by a status bit in the process state model, wherein the at least one result is provided to the at least one automation device via the process state model, and wherein the respective extension device is operable to provide an indicator via the process state model that an image has been captured and is available for further processing. ### Claim 17 An automation device comprising: a processor configured to control an industrial system based on one or more results of one or more computations performed by one or more extension devices, the one or more extension devices being operable to enable, accelerate, or enable and accelerate the one or more computations, wherein the one or more extension devices are directly connected to a camera via a peripheral interface, a processor of the one or more extension devices being configured to perform the one or more computations based at least in part on data collected from the camera by the one or more extension devices, obtain the one or more results of the one or more computations, wherein the one or more results indicate one or more states of the industrial system, wherein the one or more results are provided to the automation device via a process state model shared between the one or more extension devices and the automation device, wherein an availability of a new result of the one or more results is indicated by a status bit in the process state model, wherein the process state model is a data structure and comprises state representations of devices of the industrial system, the state representations being based on data indicative of one or more characteristics of the devices of the industrial system, and wherein the one or more extension devices are operable to provide an indicator via the process state model that an image has been captured and is available for further processing. ### Claim 18 A method for providing results of a computation in an industrial system, the method comprising: obtaining, by an extension device, one or more of the results of the computation, wherein the results indicate one or more states of the industrial system, wherein the extension device is directly connected to a camera via a peripheral interface; wherein an availability of a new result of the one or more results is indicated by a status bit in a process state model shared between the extension device and the automation device; and providing the results via the process state model to monitor, control, or monitor and control the industrial system, wherein the process state model is a data structure and comprises state representations of devices of the industrial system, the state representations being based on data indicative of one or more characteristics of the devices of the industrial system, and wherein the extension device is operable to provide an indicator via the process state model that an image has been captured and is available for further processing.