System and a process for searching massive amounts of time-series

A computer having or for coupling to a data store storing configuration attribute data and performance metric attribute data numerical values gathered as a time series from one or more performance attributes of one or more instances of one or more resource types of a system being monitored, each said numerical value of said performance metric attribute data having been mapped to a Unicode character before storage in said data store, said data store storing said configuration attribute data and said Unicode characters in a non relational database conventional file system having a special data structure which encodes metadata of each numerical value of performance metric attribute data in the structure of said data structure itself,

• said computer operative to receive a user-specified relevant interval during which data to be searched by a first search must have been gathered, and further operative to receive a user-specified first search as part of a user-specified search query also received by said computer, and said computer further operative to retrieve relevant data from said data store necessary to perform said first and second searches, said relevant data including data from a first performance or configuration attribute specified in said first search of all instances of a first resource type specified in said first search, and said first computer further operative to conduct said a first search by applying to said relevant data a first filter criteria expressed as a regular expression in said first search to find all subinterval(s) during said relevant interval during which a said first filter criteria is satisfied, said first search passing said subinterval(s) and the identities of all instances of said first resource type whose data from said first performance attribute or said first configuration attribute met said first filter criteria during one of said intervals to a user-specified second search which is part of said search query;
• and wherein said computer is further operative to conduct a said user-specified second search defined in said search query by applying a second filter criteria expressed in said second search as a regular expression only to portions of said relevant data gathered during said subinterval(s) from a second configuration attribute or a second performance attribute of instances of a resource type specified in either said first search or said second search.

A computer having or coupled to a data store storing Unicode encoded performance metric data gathered as a time-series of numerical values from each of one or more performance attributes of one or more instances of one or more resource types, each said numerical value encoded into a Unicode character so as to create a time-series of values for each performance attribute of each instance of each resource type, said data store storing said time-series of Unicode characters in a non relational database conventional file system of directories having subdirectories and files, said file system having a data structure which encodes metadata of each said Unicode character in the structure of said data structure itself, said data store also storing configuration attribute data collected over time;

• and wherein said computer is operative to conduct a first search defined in a search query of data in said data store gathered from a first configuration or performance attribute specified in said first search from all instances of a first resource type specified in said first search to find all subintervals during which a first filter condition specified as a regular expression in said first search is satisfied, said first search passing the identities of all instances of said first resource type whose configuration or performance data met said first filter condition during one of said subintervals to a second search, and assigning all said subintervals to a first time window named in said first search,
• and wherein said computer is further operative to conduct a second search defined in said search query which references said first time window name and searches only relevant data gathered during any of said subintervals from a second configuration or performance attribute specified in said second search from all instances of said first resource type passed to said second search by said first search, or from all instances of a second resource type specified in said second search which are related to an instance of said first resource type which was passed to said second search by said first search and which was gathered during said subintervals,
• and wherein said computer is operative to perform said second search by comparing said relevant data to a second filter condition specified in said second search as a regular expression, and identifying all instances which met said second filter condition and returning all said instances whose configuration or performance attribute data met said second filter condition;
• and wherein said computer is further operative to recognize a reverse mapping syntax building block included in either said first or second filter condition of said first or second search in association with a number or numerical range and convert said number or numerical range to a Unicode character or range of Unicode characters prior to comparing said first or second filter condition to configuration or performance attribute data.

A process comprising:

• using a computer to perform time window searching by performing a first search on relevant data comprising either configuration attribute data or time-series of Unicode characters stored in a non relational database flat file system and mapped from performance metric data collected as a time-series during a user-specified relevant interval from a first performance attribute identified in said first search of all instances of a first resource type identified in said first search, said first search carried out by said computer by comparing a first filter condition specified in said first search as a regular expression to said relevant data and noting all subinterval(s) when said relevant data met said first filter condition, and assigning said subintervals to a time window having a name specified in said first search, and passing the identity of instances whose configuration or performance attribute data met said first filter condition to a second search;
• and
• using a computer to perform said second search on new relevant data collected only during said subintervals defined by said time window named in said first search, said time window name being referred to by said second search, said new relevant data being data collected only during said subintervals from a second configuration or performance attribute identified in said second search of all instances of said first resource type passed to said second search from said first search, or from said second configuration or performance attribute of all instances of a second resource type named in said second search which are related to instances of said first resource type passed to said second search by said first search.

A process comprising:

• using a computer to perform a time window search sear on configuration attribute data or performance metric data collected as a time-series, said performance metric data numerical values having been encoded into a time-series of Unicode characters and stored in a non relational database flat file system comprised of directories, subdirectories and files, said searching accomplished by performing a first search on relevant data collected during a relevant time interval from a first attribute identified in said first search of all instances of a first resource type identified in said first search, said first search carried out by said computer by comparing a first filter condition specified in said first search as a regular expression to said relevant data and noting all subinterval(s) during said relevant time interval when data collected from instances of said first resource type met said first filter condition, and assigning said subintervals to a time window having a name specified in said first search, and passing the identity of instances whose data met said first filter condition and all said subintervals to a second search;
• and
• using said computer to perform said second search on new relevant data collected only during said subintervals identified by said time window named in said first search, said time window name being referred to by said second search, said new relevant data being data collected only during said subintervals and only from a second attribute identified in said second search of all instances said first resource type passed to said second search from said first search, or collected from a second attribute identified in said second search of all instances of a second resource type specified in said second search which are related to instances of said first resource type passed to said second search by said first search.