# US 8,751,499 B1 · Variable representative sampling under resource constraints > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 8,751,499 B1 - Document ID: US8751499B1 - Status: Granted - Kind: B1 - Application number: 13/747,153 - Filed: 2013-01-22 - Granted: 2014-06-10 - Earliest claimed: 2013-01-22 - Assignees: Splunk Inc. - Applicants: Splunk Inc. - Inventors: R. David Carasso; Micah James Delfino - Web page: https://patentgrep.com/patent/US8751499B1 ## CPC - G06F 16/254: {Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses} - G06F 3/0482: Interaction with lists of selectable items, e.g. menus - G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) - G06F 7/24: Sorting, i.e. extracting data from one or more carriers, rearranging the data in numerical or other ordered sequence, and rerecording the sorted data on the original carrier or on a different carrier or set of carriers {sorting methods in general}(G06F7/36 takes precedence) - G06F 16/287: {Visualization; Browsing} - G06F 16/35: Clustering; Classification - G06F 16/904: Browsing; Visualisation therefor (for navigating the web G06F16/954; browsing optimisation for the web G06F16/957) - G06F 3/0488: using a touch-screen or digitiser, e.g. input of commands through traced gestures - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Embodiments are directed towards generating a representative sampling as a subset from a larger dataset that includes unstructured data. A graphical user interface enables a user to provide various data selection parameters, including specifying a data source and one or more subset types desired, including one or more of latest records, earliest records, diverse records, outlier records, and/or random records. Diverse and/or outlier subset types may be obtained by generating clusters from an initial selection of records obtained from the larger dataset. An iteration analysis is performed to determine whether a sufficient number of clusters and/or cluster types have been generated that exceed at least one threshold and when not exceeded, additional clustering is performed on additional records. From the resultant clusters, and/or other subtype results, a subset of records is obtained as the representative sampling subset. ## Claims ### Claim 1 A method that analyzes data using a network device that is operative to perform actions, comprising: receiving one or more data streams, wherein a data stream of the one or more data streams includes unstructured data, and wherein the one or more data streams includes a plurality of events separated by event boundaries defined by a rule; parsing the one or more data streams at the event boundaries to separate a plurality of events from each other, wherein each event of the plurality of events includes information from which a time stamp for the event can be extracted; selecting, at a processor, a set of events from the plurality of events, wherein the set of events is smaller than the plurality of events; performing, at the processor, a clustering algorithm on the set of events to generate a plurality of clusters, wherein each event in the set of events is assigned to a cluster in the plurality of clusters; determining at least one number, wherein the at least one number includes one or both of a first number of clusters in the plurality of clusters and a second number indicative of a size of a smallest cluster in a subset of the plurality of clusters; determining whether a number of the at least one number is less than a threshold number; if it is determined that the number is less than the threshold number, then: retrieving another set of events from the plurality of events, combining the other set of events with the set of events to produce a combined set of events, and performing the clustering algorithm on the combined set of events to generate a second plurality of clusters; selecting at the processor, a subset of events from events assigned to the plurality of clusters or the second plurality of clusters, wherein the subset of events is smaller than the set of events, wherein the subset of events is selected based on the clusters to which events in the set of events are assigned; and displaying one or more of the events in the subset of events. ### Claim 2 (depends on 1) The method of claim 1, further comprising using one or more of the events in the subset of the events in developing a field-extraction rule for extracting a value from an event to which it is applied. ### Claim 3 (depends on 1) The method of claim 1, wherein the number includes the first number of clusters in the plurality of clusters. ### Claim 4 (depends on 1) The method of claim 1, wherein the number includes the second number indicative of the size of the smallest cluster in the subset of the plurality of clusters. ### Claim 5 (depends on 1) The method of claim 1, wherein the selection of the subset of events is biased towards selecting events from a cluster of large size relative to sizes of other clusters in the plurality of clusters or the second plurality of clusters. ### Claim 6 (depends on 1) The method of claim 1, wherein the selection of the subset of events is biased towards selecting events from a cluster of small size relative to sizes of other clusters in the plurality of clusters or the second plurality of clusters. ### Claim 7 (depends on 1) The method of claim 1, further comprising sharing the subset of events with a device not performing the method. ### Claim 8 (depends on 1) The method of claim 1, further comprising saving the subset of events. ### Claim 9 (depends on 1) The method of claim 1, wherein the clustering algorithm is selected from one of a supervised or an unsupervised clustering algorithm. ### Claim 10 (depends on 1) The method of claim 1, further comprising using one or more of the events in the subset of the events in a supervised development of a field-extraction rule for extracting a value from an event to which the rule is applied. ### Claim 11 (depends on 1) The method of claim 1, further comprising using one or more of the events in the subset of the events in an unsupervised development of a field-extraction rule for extracting a value from an event to which it is applied. ### Claim 12 A non-transitory storage medium that includes a plurality of instructions for analyzing data for a plurality of nodes, wherein execution of the instructions by a processor device enables a plurality of actions, comprising: receiving one or more data streams, wherein a data stream of the one or more data streams includes unstructured data, and wherein the one or more data streams includes a plurality of events separated by event boundaries defined by a rule; parsing the one or more data streams at the event boundaries to separate a plurality of events from each other, wherein each event of the plurality of events includes information from which a time stamp for the event can be extracted; selecting a set of events from the plurality of events, wherein the set of events is smaller than the plurality of events; performing a clustering algorithm on the set of events to generate a plurality of clusters, wherein each event in the set of events is assigned to a cluster in the plurality of clusters; determining at least one number, wherein the at least one number includes one or both of a first number of clusters in the plurality of clusters and a second number indicative of a size of a smallest cluster in a subset of the plurality of clusters; determining whether a number of the at least one number is less than a threshold number; if it is determined that the number is less than the threshold number, then: retrieving another set of events from the plurality of events, combining the other set of events with the set of events to produce a combined set of events, and performing the clustering algorithm on the combined set of events to generate a second plurality of clusters; selecting a subset of events from events assigned to the plurality of clusters or the second plurality of clusters, wherein the subset of events is smaller than the set of events, wherein the subset of events is selected based on the clusters to which events in the set of events are assigned; and displaying one or more of the events in the subset of events. ### Claim 13 (depends on 12) The medium of claim 12, wherein the actions further comprise using one or more of the events in the subset of the events in developing a field-extraction rule for extracting a value from an event to which it is applied. ### Claim 14 (depends on 12) The medium of claim 12, wherein the number includes the first number of clusters in the plurality of clusters. ### Claim 15 (depends on 12) The medium of claim 12, wherein the number includes the second number indicative of the size of the smallest cluster in the subset of the plurality of clusters. ### Claim 16 (depends on 12) The medium of claim 12, wherein the selection of the subset of events is biased towards selecting events from a cluster of large size relative to sizes of other clusters in the plurality of clusters or the second plurality of clusters. ### Claim 17 (depends on 12) The medium of claim 12, wherein the actions further comprise sharing the subset of events with a device not performing the method. ### Claim 18 (depends on 12) The medium of claim 12, wherein the actions further comprise saving the subset of events. ### Claim 19 (depends on 12) The medium of claim 12, wherein the clustering algorithm is selected from one of a supervised or an unsupervised clustering algorithm. ### Claim 20 A system that analyzes data, the system comprising: a non-transitory data storage device storing a plurality of events generated by parsing one or more data streams that included unstructured data; and a network device, having a processor that executes at least a portion of the stored instructions to enable a plurality of actions, including: receiving one or more data streams, wherein a data stream of the one or more data streams includes unstructured data, and wherein the one or more data streams includes a plurality of events separated by event boundaries defined by a rule; parsing the one or more data streams at the event boundaries to separate a plurality of events from each other, wherein each event of the plurality of events includes information from which a time stamp for the event can be extracted; selecting a set of events from the plurality of events, wherein the set of events is smaller than the plurality of events; performing a clustering algorithm on the set of events to generate a plurality of clusters, wherein each event in the set of events is assigned to a cluster in the plurality of clusters; determining at least one number, wherein the at least one number includes one or both of a first number of clusters in the plurality of clusters and a second number indicative of a size of a smallest cluster in a subset of the plurality of clusters; determining whether a number of the at least one number is less than a threshold number; if it is determined that the number is less than the threshold number, then: retrieving another set of events from the plurality of events, combining the other set of events with the set of events to produce a combined set of events, and performing the clustering algorithm on the combined set of events to generate a second plurality of clusters; selecting a subset of events from events assigned to the plurality of clusters or the second plurality of clusters, wherein the subset of events is smaller than the set of events, wherein the subset of events is selected based on the clusters to which events in the set of events are assigned; and displaying one or more of the events in the subset of events. ### Claim 21 (depends on 20) The system of claim 20, wherein the actions further comprise using one or more of the events in the subset of the events in developing a field-extraction rule for extracting a value from an event to which it is applied. ### Claim 22 (depends on 20) The system of claim 20, wherein the number includes the first number of clusters in the plurality of clusters. ### Claim 23 (depends on 20) The system of claim 20, wherein the number includes the second number indicative of the size of the smallest cluster in the subset of the plurality of clusters. ### Claim 24 (depends on 20) The system of claim 20, wherein the selection of the subset of events is biased towards selecting events from a cluster of small size relative to sizes of other clusters in the plurality of clusters or the second plurality of clusters. ### Claim 25 (depends on 20) The system of claim 20, wherein the actions further comprise sharing the subset of events with a device not performing the method. ### Claim 26 (depends on 20) The system of claim 20, wherein the actions further comprise saving the subset of events. ### Claim 27 (depends on 20) The system of claim 20, wherein the clustering algorithm is selected from one of a supervised or an unsupervised clustering algorithm. ### Claim 28 (depends on 20) The system of claim 20, wherein the selection of the subset of events is biased towards selecting events from clusters of small size relative to sizes of other clusters in the plurality of clusters or the second plurality of clusters. ### Claim 29 (depends on 20) The system of claim 20, wherein the actions further comprise using one or more of the events in the subset of the events in an unsupervised development of a field-extraction rule for extracting a value from an event to which the rule is applied. ### Claim 30 (depends on 20) The system of claim 20, wherein the selection of the subset of events is biased towards selecting events from a cluster of large size relative to sizes of other clusters in the plurality of clusters or the second plurality of clusters.