# US 11,308,049 B2 · Method and system for adaptively removing outliers from data used in training of predictive models > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,308,049 B2 - Document ID: US11308049B2 - Status: Granted - Kind: B2 - Application number: 15/707,454 - Filed: 2017-09-18 - Published: 2018-03-22 - Granted: 2022-04-19 - Earliest claimed: 2016-09-16 - Assignees: Oracle International Corporation - Applicants: Oracle International Corporation - Inventors: Yaser I. Suleiman; Michael Zoll; Subhransu Basu; Angelo Pruscino; Wolfgang Lohwasser; Wataru Miyoshi; Thomas Breidt; Thomas Herter; Klaus Thielen; Sahil Kumar - Web page: https://patentgrep.com/patent/US11308049B2 ## CPC - G06F 16/215: Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors - G06F 11/3409: {for performance assessment} - G06F 11/3452: {Performance evaluation by statistical analysis} - G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting - G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks - G06N 20/20: Ensemble learning - H04L 41/069: using logs of notifications; Post-processing of notifications - H04L 41/142: using statistical or mathematical methods - G06N 20/00: Machine learning - H04L 41/0695: the faulty arrangement being the maintenance, administration or management system - H04L 41/0823: characterised by the purposes of a change of settings, e.g. optimising configuration for enhancing reliability (for optimising operational conditions of wireless networks H04W24/02) - H04L 43/02: Capturing of monitoring data - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract Described is an improved approach to remove data outliers by filtering out data correlated to detrimental events within a system. One or more detrimental even conditions are defined to identify and handle abnormal transient states from collected data for a monitored system. ## Claims ### Claim 1 A method for generating training data for a machine learning system, comprising: collecting first data pertaining to a first computing node and second data pertaining to a second computing node in a clustered database system; identifying an abnormal state of the first computing node in the clustered database system; identifying one or more filter conditions that correspond to the abnormal state, wherein the one or more filter conditions are defined with a declaration that includes a signal name, a scope, and an apply period, the apply period includes a filter range that comprises an ordered pair, and the ordered pair comprises a first component indicative of a first number of datapoints generated before the one or more filter conditions for the abnormal state turn true and a second component indicative of a second number of datapoints generated after the one or more filter conditions for the abnormal state turn false; prior to training a predictive model with training data for the clustered database system, performing a set of acts that comprises: filtering out a first portion of the first data pertaining to the abnormal state of the first computing node; and after the second computing node has been determined to be in a normal state during a time period corresponding to the abnormal state of the first computing node, filtering out a second portion of the second data pertaining to the second computing node within the time period during which the second computing node is determined to be in the normal state on a basis that the time period corresponds to the abnormal state of the first computing node; and training the predictive model at least by providing the predictive model with the training data from which the first portion of the first data and the second portion of the second data have been filtered out. ### Claim 2 (depends on 1) The method of claim 1, further comprising removing at least one of a preceding datapoint and a subsequent datapoint from the first portion of the first data or the second portion of the second data to generate the training data. ### Claim 3 (depends on 1) The method of claim 1, wherein a scope is defined for one or more filter conditions that correspond to the abnormal state for filtering out the first portion or the second portion, the scope identifies an extent to which filtering is applied upon detection of the abnormal state, and the extent comprises a number of computing nodes whose data is subject to the one or more filter condition. ### Claim 4 (depends on 3) The method of claim 3, wherein the scope corresponds to at least one of a global scope where filtering is applied to all levels of a component hierarchy of the clustered database system, a local scope where filtering is applied only to a local level of the second computing node at which the abnormal state occurs, or an intermediate scope where filtering is applied to multiple computing nodes or multiple instances but not all computing nodes or all instances of the clustered database system. ### Claim 5 (depends on 1) The method of claim 1, wherein filtering is applied as a pre-processing action when collecting the first and the second data and before persistently storing at least some of the data that has been collected for training the predictive model. ### Claim 6 (depends on 1) The method of claim 1, wherein the training data comprises only normal operating data when the clustered database system operates in a normal operation state and also excludes the second portion of the second data representing the second computing node in the normal operation state, the abnormal state corresponds to a system maintenance task or a failure, and the predictive model is applied to perform health monitoring of the clustered database cluster. ### Claim 7 (depends on 1) The method of claim 1, wherein the time period within which the second computing node is determined to be in the normal state corresponds to the abnormal state of the first computing node in that the time period includes at least one of a first time period before detection of the abnormal state, a second time period during which the first computing node is determined to be in the abnormal state, or a third time period after a determination that the abnormal state has ceased to exist. ### Claim 8 A system for generating training data for a machine learning system, comprising: a processor; and a memory for holding programmable code, wherein the programmable code includes instructions which, when executed by the processor, cause the processor to perform a set of acts for collecting data from a clustered database system, the set of acts comprising: collecting first data pertaining to a first computing node and second data pertaining to a second computing node in the clustered database system; identifying an abnormal state of the first computing node in the clustered database system; identifying one or more filter conditions that correspond to the abnormal state, wherein the one or more filter conditions are defined with a declaration that includes a signal name, a scope, and an apply period, the apply period includes a filter range that comprises an ordered pair, and the ordered pair comprises a first component indicative of a first number of datapoints generated before the one or more filter conditions for the abnormal state turn true and a second component indicative of a second number of datapoints generated after the one or more filter conditions for the abnormal state turn false; prior to training a predictive model with training data for the clustered database system, performing a set of acts that comprises: filtering out a first portion of the first data pertaining to the abnormal state of the first computing node; and after the second computing node has been determined to be in a normal state during a time period corresponding to the abnormal state of the first computing node, filtering out a second portion of the second data pertaining to the second computing node within the time period during which the second computing node is determined to be in the normal state on a basis that the time period corresponds to the abnormal state of the first computing node; and training the predictive model at least by providing the predictive model with the training data from which the first portion of the first data and the second portion of the second data have been filtered out. ### Claim 9 (depends on 8) The system of claim 8, wherein the programmable code further includes instructions for removing at least one of a preceding datapoint and an antecedent datapoint from the first portion of the first data or the second portion of the second data to generate the training data. ### Claim 10 (depends on 8) The system of claim 8, wherein a scope of defined for one or more filter conditions that correspond to the abnormal state for filtering out the first portion or the second portion, the scope identifies an extent to which filtering is applied upon detestation of the abnormal state, and the extent comprises a number of computing nodes whose data is subject to the one or more filter condition. ### Claim 11 (depends on 10) The system of claim 10, wherein the scope corresponds to at least one of a global scope where filtering is applied to all levels of a component hierarchy of the clustered database system, a local scope where filtering is applied only to a local level of the second computing node at which the abnormal state occurs, or an intermediate scope where filtering is applied to multiple computing nodes or multiple instances but not all computing nodes or all instance of the clustered database system. ### Claim 12 (depends on 8) The system of claim 8, wherein filtering is applied as a pre-processing action when collecting the first and the second data and before persistently storing at least some of the data that has been collected for training the predictive model. ### Claim 13 (depends on 8) The system of claim 8, wherein the training data comprises only normal operating data when the clustered database system operates in a normal operation state and also excludes the second portion of the second data representing the second computing node in the normal operation state, the abnormal state corresponds to a system maintenance task or a failure, and the predictive model is applied to perform health monitoring of a database cluster. ### Claim 14 A computer program product embodied on a non-transitory computer readable medium having stored thereon a sequence of instructions which, when executed by a processor, causes the processor to perform a set of acts, the set of acts comprising: collecting first data pertaining to a first computing node and second data pertaining to a second computing node in a clustered database system; identifying an abnormal state of the first computing node in the clustered database system; identifying one or more filter conditions that correspond to the abnormal state, wherein the one or more filter conditions are defined with a declaration that includes a signal name, a scope, and an apply period, the apply period includes a filter range that comprises an ordered pair, and the ordered pair comprises a first component indicative of a first number of datapoints generated before the one or more filter conditions for the abnormal state turn true and a second component indicative of a second number of datapoints generated after the one or more filter conditions for the abnormal state turn false; prior to training a predictive model with training data for the clustered database system, performing a set of acts that comprises: filtering out a first portion of the first data pertaining to the abnormal state of the first computing node; and after the second computing node has been determined to be in a normal state during a time period corresponding to the abnormal state of the first computing node, filtering out a second portion of the second data pertaining to the second computing node within a time period during which the second computing node is determined to be in a normal state on a basis that the time period corresponds to the abnormal state of the first computing node; and training the predictive model at least by providing the predictive model with the training data from which the first portion of the first data and the second portion of the second data have been filtered out. ### Claim 15 (depends on 14) The computer program product of claim 14, wherein the sequence of instructions, when executed by the processor, cause the processor to perform the set of acts that further comprises removing at least one of a preceding datapoint and an antecedent datapoint from the first portion of the first data or the second portion of the second data to generate the training data. ### Claim 16 (depends on 14) The computer program product of claim 14, wherein a scope of defined for one or more filter conditions that correspond to the abnormal state for filtering out the first portion or the second portion, the scope identifies an extent to which filtering is applied upon detestation of the abnormal state, and the extent comprises a number of computing nodes whose data is subject to the one or more filter condition. ### Claim 17 (depends on 16) The computer program product of claim 16, wherein the scope corresponds to at least one of a global scope where filtering is applied to all levels of a component hierarchy of the clustered database system, a local scope where filtering is applied only to a local level of the second computing node at which the abnormal state occurs, or an intermediate scope where filtering is applied to multiple computing nodes or multiple instances but not all computing nodes or all instances of the clustered database system. ### Claim 18 (depends on 14) The computer program product of claim 14, wherein filtering is applied as a pre-processing action when collecting the first and the second data and before persistently storing at least some of the data that has been collected for training the predictive model. ### Claim 19 (depends on 14) The computer program product of claim 14, wherein the training data comprises only normal operating data when the clustered database system operates in a normal operation state and also excludes the second portion of the second data representing the second computing node in the normal operation state, the abnormal state corresponds to a system maintenance task or a failure of the second computing node, and the predictive model is applied to perform health monitoring of a database cluster.