# US 2021/0342193 A1 · Multi-cluster container orchestration > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0342193 A1 - Document ID: US20210342193A1 - Status: Published application - Kind: A1 - Application number: 16/862,319 - Filed: 2020-04-29 - Published: 2021-11-04 - Earliest claimed: 2020-04-29 - Assignees: International Business Machines Corporation - Applicants: International Business Machines Corporation - Inventors: Vishal ANAND - Web page: https://patentgrep.com/patent/US20210342193A1 ## CPC - G06F 9/5038: {considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration (scheduling strategies G06F9/4881 and subgroups)} - G06F 9/5072: {Grid computing} - G06F 9/5077: {Logical partitioning of resources; Management or configuration of virtualized resources (specific details on emulation or internal functioning of virtual machines G06F9/455)} - G06F 11/3006: {where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems (multiprogramming arrangements G06F9/46; allocation of resources G06F9/50)} - G06F 11/3409: {for performance assessment} - G06F 11/3419: {by assessing time} - G06F 11/3447: {Performance evaluation by modeling} - G06N 20/00: Machine learning - G06F 2201/81: Threshold - G06F 2209/5019: Workload prediction - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract Methods, computer program products, and systems are presented. The method computer program products, and systems can include, for instance: receiving, by a manager node, from a plurality of compute nodes metrics data, the manager node and the plurality of compute nodes defining a first local cluster of a first computing environment, wherein nodes of the compute nodes defining the first local cluster have running thereon container based applications, wherein a first container based application runs on a first compute node of the plurality of compute nodes defining the first local cluster, and wherein a second compute node of the plurality of compute nodes defining the first local cluster runs a second container based application; wherein the manager node has received from an orchestrator availability data specifying a set of compute nodes available for hosting the first application. ## Claims ### Claim 1 A computer implemented method comprising: receiving, by a manager node, from a plurality of compute nodes metrics data, the manager node and the plurality of compute nodes defining a first local cluster of a first computing environment, wherein nodes of the compute nodes defining the first local cluster have running thereon container based applications, wherein a first container based application runs on a first compute node of the plurality of compute nodes defining the first local cluster, and wherein a second compute node of the plurality of compute nodes defining the first local cluster runs a second container based application; wherein the manager node has received from an orchestrator availability data specifying a set of compute nodes available for hosting the first application, the set of compute nodes including a certain compute node, the certain compute node being located in a second local cluster of a second computing environment; in response to the first container based application terminating, examining the availability data specifying the set of compute nodes available for hosting the first application; selecting, in dependence on the examining, the certain compute node for hosting the first container based application; and sending, by the manager node, command data for respawning the first container based application on the certain compute node. ### Claim 2 (depends on 1) The computer implemented method of claim 1, wherein the manager node receives from the orchestrator a trained predictive model, and wherein the selecting is in dependence on a result of the manager node querying the trained predictive model. ### Claim 3 (depends on 1) The computer implemented method of claim 1, wherein the manager node receives from the orchestrator a trained predictive model, and wherein the selecting is in dependence on a result of the manager node querying the trained predictive model, wherein the trained predictive model is trained with use of historical data obtained by the orchestrator from multiple clusters, the multiple clusters including clusters disposed in multiple computing environments other than the first computing environment. ### Claim 4 (depends on 1) The computer implemented method of claim 1, wherein the manager node has received from the orchestrator prediction data specifying predicted utilization characteristics of the first and second container based application. ### Claim 5 (depends on 1) The computer implemented method of claim 1, wherein the first compute node runs the first container based application and a third container based application. ### Claim 6 (depends on 1) The computer implemented method of claim 1, wherein the first computing environment is a private computing environment and the second computing environment is a public computing environment. ### Claim 7 (depends on 1) The computer implemented method of claim 1, wherein the first computing environment is a public cloud computing environment and the second computing environment is a private computing environment. ### Claim 8 (depends on 1) The computer implemented method of claim 1, wherein the manager node performs monitoring of traffic from the first container based application during a deployment period of the first container based application, and provides a classification of the first container based application in dependence on the monitoring, and wherein the manager node performs the selecting the certain compute node in dependence on the classification. ### Claim 9 (depends on 1) The computer implemented method of claim 1, wherein the manager node performs monitoring of traffic from the first container based application during a deployment period of the first container based application, and wherein the manager node performs the selecting the certain compute node in dependence on the monitoring. ### Claim 10 (depends on 1) The computer implemented method of claim 1, wherein the examining the availability data specifying the set of compute nodes available for hosting the first application includes subjecting to scoring nodes of the compute nodes from the first computing environment, the second computing environment, and a third computing environment, and performing the selecting of the certain compute node based on the certain compute node being the highest scored node of the nodes that are subject to scoring. ### Claim 11 (depends on 1) The computer implemented method of claim 1, wherein the manager node performs monitoring of traffic from the first container based application during a deployment period of the first container based application, and provides a classification of the first container based application as a Global Communication Container (GCC) in dependence on the monitoring indicating that (a) a level of local cluster messaging from the first container based application was below a first threshold during the deployment period, and (b) a level of computing environment external computing environment messaging from the first container based application was above a second threshold during the deployment period, and wherein the manager node performs the selecting the certain compute node in dependence on the classification by qualifying the selecting of the certain compute node as a respawn host based on the GCC classification. ### Claim 12 (depends on 1) The computer implemented method of claim 1, wherein the manager node performs monitoring of traffic from the first container based application during a deployment period of the first container based application, and provides a classification of the first container based application as a Global Communication Container (GCC) in dependence on the monitoring indicating that (a) a level of local cluster messaging from the first container based application was below a first threshold during the deployment period, and (b) a level of computing environment external computing environment messaging from the first container based application was above a second threshold during the deployment period, and wherein the manager node performs the selecting the certain compute node in dependence on the classification by qualifying the selecting of the certain compute node as a respawn host based on the GCC classification, wherein the manager node performs monitoring of traffic from the second container based application during a deployment period of the second container based application, and provides a classification of the first container based application as a Local Communication Container (LCC) in dependence on the monitoring, wherein the manager node, in response to the second container based application terminating, restricts the manager node from selecting as a response compute node a compute nodes that is external to the first cluster. ### Claim 13 (depends on 1) The computer implemented method of claim 1, wherein the examining the availability data specifying the set of compute nodes available for hosting the first application includes predicting a utilization characteristic of the first container based application, determining that the certain compute node has sufficient predicted availability to accommodate the predicted utilization, and performing the selecting of the certain compute node as a respawn host for hosting the first container based application in dependence on the determining that the certain compute node has sufficient predicted availability to accommodate the predicted utilization. ### Claim 14 (depends on 1) The computer implemented method of claim 1, wherein the examining the availability data specifying the set of compute nodes available for hosting the first application includes predicting a utilization characteristic of the first container based application with reference to a first utilization parameter, determining that the certain compute node has sufficient predicted availability with reference to a first availability parameter associated to the first utilization parameter to accommodate the predicted utilization, and performing the selecting of the certain compute node as a respawn host for hosting the first container based application in dependence on the determining that the certain compute node has sufficient predicted availability to accommodate the predicted utilization. ### Claim 15 (depends on 1) The computer implemented method of claim 1, wherein the manager node receives from the orchestrator a trained predictive model, and wherein the selecting is in dependence on a result of the manager node querying the trained predictive model, wherein the trained predictive model has been trained with use of historical data obtained by the orchestrator from multiple clusters, the multiple clusters including clusters disposed in multiple computing environments other than the first computing environment. ### Claim 16 (depends on 1) The computer implemented method of claim 1, wherein the examining the availability data specifying the set of compute nodes available for hosting the first application includes the manager node scoring respective candidate compute nodes including the certain compute node for suitability of hosting of the first container based application using a multiple factor scoring formula, and performing the selecting of the certain compute node based on the certain compute node being the highest scoring candidate compute node of the respective candidate compute nodes, wherein the manager node for providing scores to a respective candidate compute node using the scoring formula includes the manager node assigning a suitability scoring value for a first factor and a second factor, wherein the first factor is a suitability of the respective compute node for hosting the first container based application with reference to a first availability parameter of the respective compute node in relation to an associated first utilization parameter of the first container based application, wherein the second factor is a suitability of the respective compute node for hosting the first container based application with reference to a second availability parameter of the respective compute node in relation to an associated second utilization parameter of the first container based application. ### Claim 17 (depends on 1) The computer implemented method of claim 1, wherein the examining the availability data specifying the set of compute nodes available for hosting the first application includes the manager node querying a first trained predictive model for return of data specifying predicted availability of a plurality of candidate compute nodes including the certain compute node, and querying a second trained predictive model for return of data specifying predicted utilization of the first container based application, the predicted utilization being predicted utilization for the first container based application when the first container based application is respawned, the first container based application is respawned, wherein the first trained predictive model has been trained with use of historical data obtained by the orchestrator from multiple clusters, the multiple clusters including clusters disposed in multiple computing environments other than the first computing environment, wherein the second trained predictive model has been trained with use of historical data obtained by the orchestrator from multiple clusters, the multiple clusters including clusters disposed in multiple computing environments other than the first computing environment, wherein the orchestrator is external from the first computing environment, wherein the first trained predictive model has been pushed from the orchestrator to the manager node, wherein the second trained predictive model has been pushed from the orchestrator to the manager node, wherein the availability data specifying a set of compute nodes available for hosting the first application is defined by the first predictive model. ### Claim 18 (depends on 1) The computer implemented method of claim 1, wherein the examining the availability data specifying the set of compute nodes available for hosting the first application includes the manager node querying a second trained predictive model for return of data specifying predicted utilization of the first container based application, and wherein the manager node querying the second trained predictive model is performed on demand in response to the first container based application terminating. ### Claim 19 A computer program product comprising: a computer readable storage medium readable by one or more processing circuit and storing instructions for execution by one or more processor for performing a method comprising: receiving, by a manager node, from a plurality of compute nodes metrics data, the manager node and the plurality of compute nodes defining a first local cluster of a first computing environment, wherein nodes of the compute nodes defining the first local cluster have running thereon container based applications, wherein a first container based application runs on a first compute node of the plurality of compute nodes defining the first local cluster, and wherein a second compute node of the plurality of compute nodes defining the first local cluster runs a second container based application; wherein the manager node has received from an orchestrator availability data specifying a set of compute nodes available for hosting the first application, the set of compute nodes including a certain compute node, the certain compute node being located in a second local cluster of a second computing environment; in response to the first container based application terminating, examining the availability data specifying the set of compute nodes available for hosting the first application; selecting, in dependence on the examining, the certain compute node for hosting the first container based application; and sending, by the manager node, command data for respawning the first container based application on the certain compute node. ### Claim 20 A system comprising: a memory; at least one processor in communication with the memory; and program instructions executable by one or more processor via the memory to perform a method comprising: receiving, by a manager node, from a plurality of compute nodes metrics data, the manager node and the plurality of compute nodes defining a first local cluster of a first computing environment, wherein nodes of the compute nodes defining the first local cluster have running thereon container based applications, wherein a first container based application runs on a first compute node of the plurality of compute nodes defining the first local cluster, and wherein a second compute node of the plurality of compute nodes defining the first local cluster runs a second container based application; wherein the manager node has received from an orchestrator availability data specifying a set of compute nodes available for hosting the first application, the set of compute nodes including a certain compute node, the certain compute node being located in a second local cluster of a second computing environment; in response to the first container based application terminating, examining the availability data specifying the set of compute nodes available for hosting the first application; selecting, in dependence on the examining, the certain compute node for hosting the first container based application; and sending, by the manager node, command data for respawning the first container based application on the certain compute node.