Scalable centralized manager including examples of data pipeline deployment to an edge system

A platform comprising:

• a first manager instance configured to publish a received message to a communication channel, wherein the message is directed to an edge system and includes a request to deploy a data pipeline to the edge system, the data pipeline comprising a plurality of containers configured to transform data at the edge system;
• a second manager instance, the second manager instance and the first manager instance configured to form at least a portion of a distributed centralized manager, the second manager instance configured to connect to the edge system corresponding to the communication channel and to subscribe to the communication channel, wherein the second manager instance is configured to receive the received message published on the communication channel and to provide the received message to the edge system in response to receipt to cause the data pipeline to deploy on the edge system;
• and
• a containerized system configured to route the received message published by the first manager instance to the second manager instance.

A method, comprising:

• publishing, via a first manager instance of a distributed centralized manager, a received message to a communication channel, wherein the published message is directed to an edge system and includes a request to deploy a data pipeline to the edge system, the data pipeline including a plurality of containers configured to transform data at the edge system;
• routing, via a containerized system of the distributed centralized manager, the published message to a second manager instance of the distributed centralized manager;
• and
• providing, via the second manager instance, the received message to the edge system in response to receipt.

At least one non-transitory computer readable medium encoded with instructions which, when executed, cause a system to:

• receive, at a first instance of a centralized platform manager, a user message to deploy a containerized application to a multi-node containerized cluster connected to a second instance of the centralized platform manager;
• determine, by the first instance of the centralized platform manager, a communication channel for an edge system including the multi-node containerized cluster;
• and
• publish, by the first instance of the centralized platform manager, the user message on the determined communication channel for the edge system.