Back to results
US 11,455,284 B2Granted
Method and system for adaptively imputing sparse and missing data for predictive models
AssigneeOracle International Corporation
InventorsMichael Zoll, Yaser I. Suleiman, Subhransu Basu, Angelo Pruscino, Wolfgang Lohwasser, Wataru Miyoshi, Thomas Breidt, Thomas Herter, Klaus Thielen, Sahil Kumar
Life of this patent
Estimating the term…
Abstract
Described is an approach that provides an adaptive solution to missing data for machine learning systems. A gradient solution is provided that is attentive to imputation needs at each of several missingness levels. This multilevel approach treats data missingness at any of multiple severity levels while utilizing, as much as possible, the actual observed data.
Claims22
A method for imputing data for a learning system, comprising:
- collecting data from a monitored target system;
- determining one or more levels of missingness for the data collected from the monitored target system, wherein
- the one or more levels of missingness for the data at least comprise a first factor corresponding to a respective degree of missingness for each of a plurality of signal patterns in the data, wherein a signal pattern is comprised of at least one or more of a first value or one or more of a second value, the first value indicating that the data is missing and the second value indicating that the data is not missing, and a degree of missingness for a signal pattern is determined at least by identifying a percentage of the data that corresponds to the signal pattern;
- adaptively selecting a technique to correct the data for missing data, wherein the data is selectively corrected by a first technique for a relatively lower level of the missing data or selectively corrected by a second technique for a relatively greater level of the missing data, wherein detection of a first missingness level is addressed by the first technique by imputing the missing data using an iterative imputation technique to generate training data without using new data from an external data source, and detection of a second missingness level that is greater than the first missingness level is addressed by the second technique to generate the training data by adding the new data from the external data source;
- and
- performing model training with the training data.
3 more elements
A system for imputing data for a machine learning system, comprising:
- a processor;
- and
- a memory for holding programmable code,
- wherein the programmable code includes instructions for collecting data from a monitored target system;
- determining one or more levels of missingness for the data collected from the monitored target system, wherein
- the one or more levels of missingness for the data at least comprise a first factor corresponding to a respective degree of missingness for each of a plurality of signal patterns in the data, wherein a signal pattern is comprised of at least one or more of a first value or one or more of a second value, the first value indicating that the data is missing and the second value indicating that the data is not missing, and a degree of missingness for a signal pattern is determined at least by identifying a percentage of the data that corresponds to the signal pattern;
- adaptively selecting a technique to correct the data for missing data, wherein the data is selectively corrected by a first technique for a relatively lower level of the missing data or selectively corrected by a second technique for a relatively greater level of the missing data, wherein detection of a first missingness level is addressed by the first technique by imputing the missing data using an iterative imputation technique to generate training data without using new data from an external data source, and detection of a second missingness level that is greater than the first missingness level is addressed by the second technique to generate the training data by adding the new data from the external data source;
- and
- performing model training with the training data.
6 more elements
A non-transitory computer readable medium having stored thereon a sequence of instructions which, when executed by a processor, executes a method comprising:
- collecting data from a monitored target system;
- determining one or more levels of missingness for the data collected from the monitored target system, wherein
- the one or more levels of missingness for the data at least comprise a first factor corresponding to a respective degree of missingness for each of a plurality of signal patterns in the data, wherein a signal pattern is comprised of at least one or more of a first value or one or more of a second value, the first value indicating that the data is missing and the second value indicating that the data is not missing, and a degree of missingness for a signal pattern is determined at least by identifying a percentage of the data that corresponds to the signal pattern;
- adaptively selecting a technique to correct the data for missing data, wherein the data is selectively corrected by a first technique for a relatively lower level of the missing data or selectively corrected by a second technique for a relatively greater level of the missing data, wherein detection of a first missingness level is addressed by the first technique by imputing the missing data using an iterative imputation technique to generate training data without using new data from an external data source, and detection of a second missingness level that is greater than the first missingness level is addressed by the second technique to generate the training data by adding the new data from the external data source;
- and
- performing model training with the training data.
3 more elements
Dependencies are inferred from the claim text.