Methods, systems and computer readable media for coordinating publish-subscribe messaging

A system for coordinating publish-subscribe messaging, the system comprising:

• a processor;
• a memory communicatively connected to the processor;
• and
• broker implemented using the processor and the memory, wherein the broker is configured for:
• receiving, from a publisher, a first message pertaining to a first topic, wherein the first message includes a message to stop sourcing data;
• sending the first message to at least a subscriber subscribed to the first message, wherein the at least a subscriber subscribed to the first message includes at least a data source in a pipeline;
• receiving, from each of the at least a subscriber subscribed to the first message, an acknowledgment that an action related to the first message is completed, wherein the action related to the first message includes stop collecting data and finish providing data already collected;
• notifying the publisher that all acknowledgments related to the first message are received when the broker receives the acknowledgment from each of the at least a subscriber subscribed to the first message;
• and
• receiving, from the publisher, a second message based on the notification that all acknowledgments for the first message are received, wherein the second message includes a message to finish processing data.

A non-transitory computer readable medium having stored thereon executable instructions that when executed by at least one processor of at least one computer cause the at least one computer to perform steps comprising:

• receiving, from a publisher, a first message pertaining to a first topic, wherein the first message includes a message to stop sourcing data;
• sending the first message to at least a subscriber subscribed to the first message, wherein the at least a subscriber subscribed to the first message includes at least a data source in a pipelines;
• receiving, from each of the at least a subscriber subscribed to the first message, an acknowledgment that the at least a subscriber subscribed to the first message completed an action related to the first message, wherein the action related to the first message includes stop collecting data and finish providing data already collected;
• notifying the publisher that all acknowledgments related to the first message are received based on receiving the acknowledgment from each of the at least a subscriber subscribed to the first message;
• and
• receiving, from the publisher, a second message based on the notification that all acknowledgments for the first message are received, wherein the second message includes a message to finish processing data.