Systems and methods for orchestrating microservice containers interconnected via a service mesh in a multi-cloud environment based on a reinforcement learning policy

A multi-cloud service mesh orchestration platform system, comprising:

• one or more processors;
• and
• memory including instructions that, when executed by the one or more processors, cause the multi-cloud service mesh orchestration platform system to:
• receive a request to deploy an application as a service mesh application;
• instantiate the service mesh application in response to the request;
• tag the service mesh application with governance information including criteria governing how to provision computing resources from multiple Cloud Service Provider (CSP) networks for deploying and operating the service mesh application;
• partition the service mesh application into constituent microservice containers wherein each respective microservice container includes a computing environment;
• tag each microservice container with individual governance information derived from the governance information for the service mesh application;
• determining configuration information for sidecar proxies, wherein the sidecar proxies control routing between the constituent microservice containers based at least in part on the respective individual governance information of the respective microservice container;
• for each time step within a first time period, select and perform a first set of actions for deploying each microservice container using a reserved computing instance of a first CSP network of the multiple CSP networks to obtain one or more individual rewards, state transitions, and expected returns, wherein at the first time period the reserved computing instance meets one or more criteria of the governance information better than an unreserved computing instance;
• determine a reinforcement learning policy for each microservice container that maximizes a total reward for the service mesh application based on the one or more individual rewards, state transitions, and expected returns of each first set of actions selected and performed for each microservice container for each time step within the first time period;
• and
• for each time step within a second time period, select and perform a second set of actions for each microservice container based on the reinforcement learning policy for the microservice container that maximize the total reward, wherein at least one of the second set of actions migrates a microservice container of the constituent microservice containers to the unreserved computing instance of a second CSP network of the multiple CSP networks when the unreserved computing instance meets the one or more criteria of the governance information better than the reserved computing instance.

A non-transitory computer-readable storage medium including instructions that, upon being executed by one or more processors of a multi-cloud service mesh orchestration platform system, cause the multi-cloud service mesh orchestration platform system to:

• receive a request to deploy an application as a service mesh application;
• instantiate the service mesh application in response to the request;
• tag the service mesh application with governance information including criteria governing how to provision computing resources from multiple Cloud Service Provider (CSP) networks for the service mesh application;
• partition the service mesh application into constituent microservice containers wherein each respective microservice container includes a computing environment;
• tag each microservice container with individual governance information derived from the governance information for the service mesh application;
• determining configuration information for sidecar proxies, wherein the sidecar proxies control routing between the constituent microservice containers based at least in part on the respective individual governance information of the respective microservice container;
• for each time step within a first time period, select and perform a first set of action for deploying each microservice container using a reserved computing instance of a first CSP network of the multiple CSP networks to obtain one or more individual rewards, state transitions, and expected returns, wherein at the first time period the reserved computing instance meets one or more criteria of the governance information better than an unreserved computing instance;
• determine a reinforcement learning policy for each microservice container that maximizes a total reward for the service mesh application based on the one or more individual rewards, state transitions, and expected returns of each first set of actions selected and performed for each microservice container for each time step within the first time period;
• and
• for each time step within a second time period, select and perform a second set of actions for each microservice container based on the reinforcement learning policy for the microservice container that maximize the total reward, wherein at least one of the second set of actions migrates a microservice container of the constituent microservice containers to the unreserved computing instance of a second CSP network of the multiple CSP networks when the unreserved computing instance meets the one or more criteria of the governance information better than the reserved computing instance.