# US 12,183,452 B2 · Algorithm orchestration of workflows to facilitate healthcare imaging diagnostics > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,183,452 B2 - Document ID: US12183452B2 - Status: Granted - Kind: B2 - Application number: 17/102,595 - Filed: 2020-11-24 - Published: 2021-06-10 - Granted: 2024-12-31 - Earliest claimed: 2019-11-25 - Assignees: GE Precision Healthcare LLC - Applicants: GE Precision Healthcare LLC - Inventors: Garima Mathur; Dean Whitney; Paulo Gallotti; Chris Diviyanathan; Vijay Arlagadda; Garrett Rogers; Flávio Henrique Schuindt da Silva; Hrishikesh Nerurkar; Eugene Martin Salvetti, Jr. - Web page: https://patentgrep.com/patent/US12183452B2 ## CPC - G16H 40/20: for the management or administration of healthcare resources or facilities, e.g. managing hospital staff or surgery rooms - G06F 9/541: {via adapters, e.g. between incompatible applications} - G06F 11/079: {Root cause analysis, i.e. error or fault diagnosis (in a hardware test environment G06F11/22; in a software test environment G06F11/36)} - G06F 11/3476: {Data logging (G06F11/14, G06F11/2205 take precedence)} - G06F 11/3698: Environments for analysis, debugging or testing of software - G06Q 10/067: Enterprise or organisation modelling - G06Q 10/0674: {using process simulation} - G16H 30/20: for handling medical images, e.g. DICOM, HL7 or PACS - G16H 30/40: for processing medical images, e.g. editing - G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems - H04L 63/102: {Entity profiles} - G06F 11/0751: {Error or fault detection not based on redundancy (power supply failures G06F1/30; network fault management H04L41/06)} - G06F 11/0766: {Error or fault reporting or storing} - G16H 15/00: ICT specially adapted for medical reports, e.g. generation or transmission thereof - G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract Techniques for orchestrating execution of algorithms on medical images according to pre-defined workflows and techniques for managing workflow and model execution are provided. In an embodiment, a system comprises a memory that stores computer executable components and a processor that executes the computer executable components stored in the memory. The computer executable components comprise an algorithm catalog that comprises algorithm information identifying algorithms available for processing medical images, the algorithm information comprising algorithm execution instructions for executing the algorithms as web-services; and an onboarding component that adds the algorithm information to the algorithm catalog in response to reception of the algorithm information via an onboarding user interface of an algorithm management application, wherein based on inclusion of the algorithm information in the algorithm catalog, the algorithms are made available for incorporating into workflows for executing the algorithms on the medical images. ## Claims ### Claim 1 A system, comprising: a memory that stores computer-executable components; and a processor that executes the computer-executable components stored in the memory, wherein the computer executable components comprise: an algorithm catalog that comprises algorithm information identifying algorithms available for application to medical images, the algorithm information comprising algorithm execution instructions for executing the algorithms in association with invoking the algorithms at respective network accessible systems where the algorithms are stored; and a workflow creation component that provides a workflow creation application that facilitates creating a workflow made available by the system for application to one or more of the medical images, the workflow comprising workflow information defining a set of computer-executable processing steps to be applied to the one or more of the medical images, wherein the workflow creation application comprises a graphical user interface that facilitates creating the workflow using graphical nodes in association with receiving user input dragging and dropping the graphical nodes onto a workflow creation area of the graphical user interface, connecting the graphical nodes using one or more connection lines to define relationships between the graphical nodes, and configuring the graphical nodes, and wherein the graphical nodes comprise different types of graphical nodes respectively corresponding to different types of computer-executable processing steps available for processing the medical images, including a task node corresponding to an algorithm execution task, wherein based on selection of the task node, the workflow creation application provides a selectable list of the algorithms included in the algorithm catalog and integrates a selected algorithm from the selectable list into the workflow in association with importing the algorithm execution instructions corresponding to the selected algorithm into the workflow information from the algorithm catalog, and wherein the different types of graphical nodes are selected from the group consisting of: the task node, a decision node, a sub-workflow node, a wait node, a fork node, a join node and a function node. ### Claim 2 (depends on 1) The system of claim 1, wherein the computer-executable components further comprise: a workflow execution component that controls execution of the workflow on the one or more medical images in association with reception of an image processing request from a medical image provider via a network and returns workflow results to the medical imaging provider generated in association with the execution of the workflow, the workflow results comprising output data of the selected algorithm. ### Claim 3 (depends on 2) The system of claim 2, wherein at least some of the algorithms employ a different format that is not compatible with the workflow execution component, and wherein the computer-executable components further comprise: an onboarding component that adds the algorithm information to the algorithm catalog in response to reception of the algorithm information via an onboarding user interface of an algorithm management application, wherein based on inclusion of the algorithm information in the algorithm catalog, the system makes the algorithms available for incorporating into workflows created using the workflow creation application; and a translation component that generates mapping information defining a mapping operation that controls translating between the different format and a format that is compatible with the workflow execution component in association with the onboarding, and wherein the onboarding component includes the mapping information with corresponding algorithm execution instructions for the at least some of the algorithms in the algorithm catalog. ### Claim 4 (depends on 3) The system of claim 3, wherein the workflow execution component employs the mapping information in association with controlling the application of the at least some of the algorithms. ### Claim 5 (depends on 1) The system of claim 1, wherein the algorithms comprise medical image inferencing models. ### Claim 6 (depends on 3) The system of claim 3, wherein the algorithms comprise medical image inferencing models, wherein the algorithm management application further provides for adjusting one or more parameters and thresholds of the algorithms in association with receiving the algorithm information, wherein the onboarding component stores settings information with the algorithm information defining the one or more parameters and thresholds, wherein the workflow execution component applies the settings information in association the execution of the algorithms, and wherein the thresholds include a critically threshold for at least some of the medical image inferencing models. ### Claim 7 (depends on 1) The system of claim 1, wherein the computer-executable components further comprise: a simulator component that runs the workflow in a test mode on one or more test images and identifies execution errors. ### Claim 8 (depends on 1) The system of claim 1, wherein the computer-executable components further comprise: a workflow management component that provides, via an algorithm management application, access to collective workflow information defining workflows available by the system for application to the medical images, as created via the workflow creation application, and workflow management functions related to the workflows. ### Claim 9 (depends on 8) The system of claim 8, wherein the workflow management functions include an activation/deactivation function that provides for selectively activating and deactivating one or more of the workflows. ### Claim 10 A method, comprising: providing, by a system operatively coupled to a processor, an algorithm catalog that comprises algorithm information identifying algorithms available for application to medical images and algorithm execution instructions for executing the algorithms in association with invoking the algorithms at respective network accessible systems where the algorithms are stored; and providing, by the system, a workflow creation application that facilitates creating workflows made available by the system for application to the medical images, the workflows respectively comprising workflow information defining a set of computer-executable processing steps to be applied to one or more of the medical images, wherein the workflow creation application comprises a graphical user interface that facilitates creating the workflows using graphical nodes in association with receiving user input dragging and dropping the graphical nodes onto a workflow creation area of the graphical user interface, connecting the graphical nodes using one or more connection lines to define relationships between the graphical nodes, and configuring the graphical nodes, and wherein the graphical nodes comprise different types of graphical nodes respectively corresponding to different types of computer-executable processing steps available for processing the medical images, including a task node corresponding to an algorithm execution task, wherein based on selection of the task node, the workflow creation application provides a selectable list of the algorithms included in the algorithm catalog and integrates a selected algorithm from the selectable list into the workflows in association with importing the algorithm execution instructions corresponding to the selected algorithm into the workflow information from the algorithm catalog, and wherein the different types of graphical nodes are selected from the group consisting of: the task node, a decision node, a sub-workflow node, a wait node, a fork node, a join node and a function node. ### Claim 11 (depends on 10) The method of claim 10, further comprising: storing, by the system, the workflows created using the workflow creation application; receiving, by the system, image processing requests from medical imaging providers via a network; controlling, by the system, execution of the workflows on the medical images in association with reception of image processing requests; generating, by the system, workflow results based on the execution, the workflow results comprising outputs of respective algorithms integrated therein; and returning, by the system, the workflow results to the medical imaging providers via the network. ### Claim 12 (depends on 10) The method of claim 10, wherein the algorithms comprise medical image inferencing models. ### Claim 13 (depends on 10) The method of claim 10, further comprising: running, by the system, the workflows in a test mode on one or more test images; and determining, by the system, whether execution errors exist in the workflows based on the running. ### Claim 14 (depends on 10) The method of claim 10, further comprising: providing, by the system via an algorithm management application, access to collective workflow information defining the workflows and management functions related to the workflows. ### Claim 15 (depends on 14) The method of claim 14, wherein the wherein the workflow management functions include an activation/deactivation function that provides for selectively activating and deactivating one or more of the workflows. ### Claim 16 A non-transitory machine-readable storage medium, comprising executable instructions that, when executed by a processor associated with an algorithm orchestration system, facilitate performance of operations, comprising: providing an algorithm catalog that comprises algorithm information identifying algorithms available for application to medical images and algorithm execution instructions for executing the algorithms in association with invoking the algorithms at respective network accessible systems where the algorithms are stored; providing a workflow creation tool that facilitates creating workflows made available by the algorithm orchestration system for application to the medical images, the workflows respectively comprising workflow information defining a set of computer-executable processing steps to be applied to one or more of the medical images, wherein the workflow creation tool comprises a graphical user interface that facilitates creating the workflows using graphical nodes in association with receiving user input dragging and dropping the graphical nodes onto a workflow creation area of the graphical user interface, connecting the graphical nodes using one or more connection lines to define relationships between the graphical nodes, and configuring the graphical nodes, and wherein the graphical nodes comprise different types of graphical nodes respectively corresponding to different types of computer-executable processing steps available for processing the medical images, including a task node corresponding to an algorithm execution task, wherein based on selection of the task node, the workflow creation tool provides a selectable list of the algorithms included in the algorithm catalog and integrates a selected algorithm from the selectable list into the workflows in association with importing the algorithm execution instructions corresponding to the selected algorithm into the workflow information from the algorithm catalog, and wherein the different types of graphical nodes are selected from the group consisting of: the task node, a decision node, a sub-workflow node, a wait node, a fork node, a join node and a function node. ### Claim 17 (depends on 16) The non-transitory machine-readable storage medium of claim 16, the operations further comprising: storing the workflows created using the workflow creation tool; receiving image processing requests from medical imaging providers via a network; controlling execution of the workflows on the medical images in association with reception of the image processing requests; generating workflow results based on the execution, the workflow results comprising one or more of outputs of respective algorithms incorporated therein; and returning the workflow results to the medical imaging providers via the network. ### Claim 18 (depends on 2) The system of claim 2, wherein the workflow execution component retrieves the one or more medical images from one or more source storage systems and loads the one or more medical images into a local storage system based on reception of the image processing request and a determination that the workflow is applicable to the one or more medical images, and wherein the execution component controls the execution of the workflow on the one or more medical images in association with accessing the one or more medical images from the local storage system. ### Claim 19 (depends on 1) The system of claim 1, wherein the execution instructions control the application of the algorithms to the medical images as web-services. ### Claim 20 (depends on 1) The system of claim 1, wherein the execution instructions control the application of the algorithms to the medical images as Kubernetes jobs. ### Claim 21 (depends on 1) The system of claim 1, wherein the different types of graphical nodes further include other types of nodes corresponding to other types of the different types of the computer-executable steps available for processing the medical images, including at least one of, a medical image filtering step, a data retrieval step, a data moving step, or a data deletion step. ### Claim 22 (depends on 1) The system of claim 1, wherein the different types of graphical nodes are visually distinctive from one another. ### Claim 23 (depends on 1) The system of claim 1, wherein the different types of graphical nodes comprise different shapes or symbols. ### Claim 24 (depends on 1) The system of claim 1, wherein the different types of nodes respectively correspond to different computer-executable code.