Controlling telemetry consumers in a telemetry subscription architecture for cluster file systems

A method of processing telemetry data in a cluster network having a plurality of telemetry producers each periodically generating metric datasets, comprising:

• processing the telemetry data through a hardware-based telemetry pipeline, wherein the cluster network comprises a Santorini network processing containerized data utilizing a Kubernetes-based framework and that stores filesystem metadata on a distributed key value store and file data in an object store and exposes a global namespace that is a union of all namespaces in all domains, and further wherein the plurality of nodes each contain a plurality of pods performing network functions and generating the telemetry data for transmission to the subscribing consumers, and yet further wherein the telemetry data comprises data generated periodically by each producer upon operation in the cluster network, and consists of performance data, topology information, alerts, security states, and service features;
• first receiving schema definitions of a metric dataset of the telemetry data produced by a telemetry producer of the telemetry producers;
• storing the schema definitions in a telemetry catalog accessed by a telemetry transmitter;
• second receiving a list of consumers selected to receive telemetry data including the metric dataset;
• accepting or rejecting the list consumers in a first validation;
• and
• allowing transmission of the metric dataset to any of the validated consumers if the list is accepted;
• and
• blocking transmission of the metric dataset by any of the consumers if the list is rejected.

A system processing telemetry data in a cluster network having nodes each containing a plurality of pods, the system comprising:

• a hardware-based telemetry pipeline processing the telemetry data through, wherein the cluster network comprises a Santorini network processing containerized data utilizing a Kubernetes-based framework and that stores filesystem metadata on a distributed key value store and file data in an object store and exposes a global namespace that is a union of all namespaces in all domains, and further wherein the plurality of nodes each contain a plurality of pods performing network functions and generating the telemetry data for transmission to the subscribing consumers, and yet further wherein the telemetry data comprises data generated periodically by each producer upon operation in the cluster network, and consists of performance data, topology information, alerts, security states, and service features;
• a telemetry catalog storing schema of telemetry data generated as metric datasets by telemetry producers in the network;
• a telemetry transmitter component receiving a list of consumers selected to receive a metric dataset from a producer;
• a telemetry collector component accepting or rejecting the consumers to allow or block them from receiving the metric dataset;
• an interface transmitting a decision of acceptance or rejection to a telemetry library of the producer;
• and
• accepting the metric dataset from the producer for storage and receipt by the consumers if the decision is acceptance.