Variable representative sampling under resource constraints

A non-transitory storage medium that includes a plurality of instructions for analyzing data for a plurality of nodes, wherein execution of the instructions by a processor device enables a plurality of actions, comprising:

• receiving one or more data streams, wherein a data stream of the one or more data streams includes unstructured data, and wherein the one or more data streams includes a plurality of events separated by event boundaries defined by a rule;
• parsing the one or more data streams at the event boundaries to separate a plurality of events from each other, wherein each event of the plurality of events includes information from which a time stamp for the event can be extracted;
• selecting a set of events from the plurality of events, wherein the set of events is smaller than the plurality of events;
• performing a clustering algorithm on the set of events to generate a plurality of clusters, wherein each event in the set of events is assigned to a cluster in the plurality of clusters;
• determining at least one number, wherein the at least one number includes one or both of a first number of clusters in the plurality of clusters and a second number indicative of a size of a smallest cluster in a subset of the plurality of clusters;
• determining whether a number of the at least one number is less than a threshold number;
• if it is determined that the number is less than the threshold number, then:
• retrieving another set of events from the plurality of events,
• combining the other set of events with the set of events to produce a combined set of events, and
• performing the clustering algorithm on the combined set of events to generate a second plurality of clusters;
• selecting a subset of events from events assigned to the plurality of clusters or the second plurality of clusters, wherein the subset of events is smaller than the set of events, wherein the subset of events is selected based on the clusters to which events in the set of events are assigned;
• and
• displaying one or more of the events in the subset of events.

A system that analyzes data, the system comprising:

• a non-transitory data storage device storing a plurality of events generated by parsing one or more data streams that included unstructured data;
• and
• a network device, having a processor that executes at least a portion of the stored instructions to enable a plurality of actions, including:
• receiving one or more data streams, wherein a data stream of the one or more data streams includes unstructured data, and wherein the one or more data streams includes a plurality of events separated by event boundaries defined by a rule;
• parsing the one or more data streams at the event boundaries to separate a plurality of events from each other, wherein each event of the plurality of events includes information from which a time stamp for the event can be extracted;
• selecting a set of events from the plurality of events, wherein the set of events is smaller than the plurality of events;
• performing a clustering algorithm on the set of events to generate a plurality of clusters, wherein each event in the set of events is assigned to a cluster in the plurality of clusters;
• determining at least one number, wherein the at least one number includes one or both of a first number of clusters in the plurality of clusters and a second number indicative of a size of a smallest cluster in a subset of the plurality of clusters;
• determining whether a number of the at least one number is less than a threshold number;
• if it is determined that the number is less than the threshold number, then:
• retrieving another set of events from the plurality of events,
• combining the other set of events with the set of events to produce a combined set of events, and
• performing the clustering algorithm on the combined set of events to generate a second plurality of clusters;
• selecting a subset of events from events assigned to the plurality of clusters or the second plurality of clusters, wherein the subset of events is smaller than the set of events, wherein the subset of events is selected based on the clusters to which events in the set of events are assigned;
• and
• displaying one or more of the events in the subset of events.