Methods, systems, and computer readable mediums for performing metadata-driven data collection

A system comprising:

• at least one processor and memory;
• and
• a data collection engine implemented using the at least one processor, wherein the data collection engine is programmed for:
• receiving a request for system status data for a plurality of components of a distributed computing system while the distributed computing system is in operation, wherein the request includes metadata specifying a data collection sequence for collecting component-level system status data, and wherein the plurality of components of the distributed computing system includes a plurality of compute components, a plurality of network components, and a plurality of storage components;
• obtaining, using the metadata, the component-level system status data by querying a plurality of protocol-based data collectors in an order, one after the other, as specified by the data collection sequence specified by the metadata, wherein the metadata specifies, for at least some of the component-level system status data, the data collection sequence so that at least some earlier-executing data collectors are assured to execute before some later-executing data collectors;
• and
• assembling the component-level system status data into assembled status data and storing the assembled status data in the memory;
• wherein obtaining the component-level system status data comprises querying a first protocol-based data collector and then, after querying the first protocol-based data collector, querying a second protocol-based data collector according to the data collection sequence and using the response from the first protocol-based data collector, and wherein assembling the component-level system status data comprises using a first attribute received from the first protocol-based data collector for updating an assembled object with a second attribute received from the second protocol-based data collector;
• wherein querying the first protocol-based data collector comprises receiving the first attribute and a common attribute, wherein querying the second protocol-base data collector comprises receiving the second attribute and the common attribute, and wherein updating an assembled object comprises associating the first and second attributes and the common attribute in the assembled object;
• and
• wherein the data collection engine is programmed for receiving an update to the metadata and repeating the obtaining and assembling of the component-level system status data using the update to the metadata without recompiling executable code of the protocol-based data collectors, and wherein the common attribute is a unique identifier of a virtual machine, and wherein updating the assembled object comprises combining the first attribute and the second attribute into a report about the virtual machine.

A non-transitory computer readable medium storing executable instructions that, when executed by at least one processor of a computer system, cause the computer system to perform operations comprising:

• receiving a request for system status data for a plurality of components of a distributed computing system while the distributed computing system is in operation, wherein the request includes metadata specifying a data collection sequence for collecting component-level system status data, and wherein the plurality of components of the distributed computing system includes a plurality of compute components, a plurality of network components, and a plurality of storage components;
• obtaining, using the metadata, the component-level system status data by querying a plurality of protocol-based data collectors in an order, one after the other, as specified by the data collection sequence specified by the metadata, wherein the metadata specifies, for at least some of the component-level system status data, the data collection sequence so that at least some earlier-executing data collectors are assured to execute before some later-executing data collectors;
• and
• assembling the component-level system status data into assembled status data and storing the assembled status data in memory of the computer system;
• wherein obtaining the component-level system status data comprises querying a first protocol-based data collector and then, after querying the first protocol-based data collector, querying a second protocol-based data collector according to the data collection sequence and using the response from the first protocol-based data collector, and wherein assembling the component-level system status data comprises using a first attribute received from the first protocol-based data collector for updating an assembled object with a second attribute received from the second protocol-based data collector;
• wherein querying the first protocol-based data collector comprises receiving the first attribute and a common attribute, wherein querying the second protocol-base data collector comprises receiving the second attribute and the common attribute, and wherein updating an assembled object comprises associating the first and second attributes and the common attribute in the assembled object;
• and
• wherein the operations comprise receiving an update to the metadata and repeating the obtaining and assembling of the component-level system status data using the update to the metadata without recompiling executable code of the protocol-based data collectors, and wherein the common attribute is a unique identifier of a virtual machine, and wherein updating the assembled object comprises combining the first attribute and the second attribute into a report about the virtual machine.