# US 12,217,145 B2 · Continuously learning, stable and robust online machine learning system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,217,145 B2
- Document ID: US12217145B2
- Status: Granted
- Kind: B2
- Application number: 17/695,787
- Filed: 2022-03-15
- Published: 2022-06-30
- Granted: 2025-02-04
- Earliest claimed: 2018-03-06
- Assignees: Tazi AI Systems, Inc.
- Applicants: Tazi AI Systems, Inc.
- Inventors: Tanju Cataltepe
- Web page: https://patentgrep.com/patent/US12217145B2

## CPC

- G06N 20/20: Ensemble learning
- G05B 13/028: {using expert systems only}
- G05B 23/0221: {Preprocessing measurements, e.g. data collection rate adjustment; Standardization of measurements; Time series or signal analysis, e.g. frequency analysis or wavelets; Trustworthiness of measurements; Indexes therefor; Measurements using easily measured parameters to estimate parameters difficult to measure; Virtual sensor creation; De-noising; Sensor fusion; Unconventional preprocessing inherently present in specific fault detection methods like PCA-based methods}
- G05B 23/0229: {knowledge based, e.g. expert systems; genetic algorithms}
- G06F 3/165: {Management of the audio stream, e.g. setting of volume, audio stream path}
- G06F 16/24568: {Data stream processing; Continuous queries}
- G06F 18/10: Pre-processing; Data cleansing
- G06F 18/15: Statistical pre-processing, e.g. techniques for normalisation or restoring missing data
- G06F 18/2115: by evaluating different subsets according to an optimisation criterion, e.g. class separability, forward selection or backward elimination
- G06F 18/2178: {based on feedback of a supervisor}
- G06F 18/23: Clustering techniques
- G06F 18/40: Software arrangements specially adapted for pattern recognition, e.g. user interfaces or toolboxes therefor
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/045: Combinations of networks
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/08: Learning methods
- G06N 3/082: modifying the architecture, e.g. adding, deleting or silencing nodes or connections
- G06N 3/0895: Weakly supervised learning, e.g. semi-supervised or self-supervised learning
- G06N 3/09: Supervised learning
- G06N 3/091: Active learning
- G06N 3/098: Distributed learning, e.g. federated learning
- G06N 3/0985: Hyperparameter optimisation; Meta-learning; Learning-to-learn
- G06N 5/01: Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
- G06N 5/043: Distributed expert systems; Blackboards
- G06N 5/045: Explanation of inference; Explainable artificial intelligence [XAI]; Interpretable artificial intelligence
- G06N 7/00: Computing arrangements based on specific mathematical models
- G06N 20/00: Machine learning
- G06V 10/28: Quantising the image, e.g. histogram thresholding for discrimination between background and foreground patterns
- G06V 10/70: using pattern recognition or machine learning (optical pattern recognition or electronic computations therefor G06V10/88)
- G06V 10/72: Data preparation, e.g. statistical preprocessing of image or video features
- G06V 10/77: Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V 10/778: Active pattern-learning, e.g. online learning of image or video features
- G06V 10/7784: {based on feedback from supervisors}
- G06V 10/80: Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level (multimodal speaker identification or verification G10L17/10)
- G06V 10/803: {of input or preprocessed data}
- G06T 2207/20081: Training; Learning
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

An Online Machine Learning System (OMLS) including an Online Preprocessing Engine (OPrE) configured to (a) receive streaming data including an instance comprising a vector of inputs, the vector of inputs comprising a plurality of continuous or categorical features; (b) discretize features; (c) impute missing feature values; (d) normalize features; and (e) detect drift or change in features; an Online Feature Engineering Engine (OFEE) configured to produce features; and an Online Robust Feature Selection Engine (ORFSE) configured to evaluate and select features; an Online Machine Learning Engine (OMLE) configured to incorporate and utilize one or more machine learning algorithms or models utilizing features to generate a result, and capable of incorporating and utilizing multiple different machine learning algorithms or models, wherein each of the OMLE, the OPrE, the OFEE, and the ORFSE are continuously communicatively coupled to each other, and wherein the OMLS is configured to perform continuous online machine learning.

## Claims

### Claim 1

An Online Machine Learning System (OMLS) implemented on one or more electronic devices comprising one or more processors, the system comprising:
an Online Preprocessing Engine (OPrE) stored on a non-transitory computer readable memory and implemented using the one or more processors and configured to (a) receive streaming data including a set of instances, each instance of the set of instances comprising one or more vectors of inputs that include a plurality of continuous or categorical features produced externally or internally by the OMLS or by actions of one or more of users of the OMLS; (b) discretize features; (c) impute missing feature values; (d) normalize features; (e) detect drift or change in features; and (f) detect drift or change in labels for the set of instances, wherein each of the labels is an actual label or a predicted label;
an Online Feature Engineering Engine (OFEE) stored on a non-transitory computer readable memory and implemented using the one or more processors and configured to produce features including engineered features;
an Online Robust Feature Selection Engine (ORFSE) stored on a non-transitory computer readable memory and implemented using the one or more processors, and configured to evaluate and select features by computing a relevance value of one of the features based on a computation of a degree of correlation between a first feature and a second feature using at least one of (i) mutual information-based label relevances between features, (ii) distributional similarities between features, or (iii) similarity of change in distributions of two features to reduce correlations and redundancies between selected features and performing a statistical significance test to determine a significance of the relevance value that is computed using at least one of (i), (ii), or (iii); and
an Online Machine Learning Engine (OMLE) stored on a non-transitory computer readable memory and implemented using the one or more processors, and configured to incorporate and utilize one or more machine learning algorithms or models utilizing features selected by the ORFSE, to generate a result, and capable of incorporating and utilizing multiple different machine learning algorithms or models,
wherein each of the OMLE, the OPrE, the OFEE, and the ORFSE are continuously communicatively coupled to each other,
wherein the OPrE, which is capable of detecting the drift or change in features, is capable of detecting the drift or changes in the plurality of continuous or categorical features, the discretized features, and the engineered features, and
wherein the OMLS is configured to perform continuous online machine learning.

### Claim 2 (depends on 1)

The system of claim 1, wherein the OMLS is further configured to automatically select features, either continuously or triggered by one or more events, based on machine learning performances and business benefit performances of the OMLS, wherein the one or more events comprises a change in inputs or actual or predicted outputs that exceeds a specified threshold.

### Claim 3 (depends on 1)

The system of claim 1, wherein the OMLS is further configured to select and adaptively combine one or more machine learning algorithms, models or other components based on machine learning performances and business benefit performances.

### Claim 4 (depends on 1)

The system of claim 1, wherein the OMLS is further configured to implement automated feature processing methods, comprising at least one of transformation, engineering, deletion or imputation, either continuously or triggered by one or more events, based on machine learning performances and business benefit performances of the OMLS, wherein the one or more events comprise a change in inputs or actual or predicted outputs that exceeds a specified threshold.

### Claim 5 (depends on 1)

The system of claim 1, wherein the OMLS is further configured to automatically select hyperparameters for one or more of components of the OMLS, either continuously or triggered by one or more events based on machine learning performance and business benefit performance of the OMLS, wherein the one or more events comprise a change in inputs or actual or predicted outputs that exceeds a specified threshold.

### Claim 6 (depends on 1)

The system of claim 1, wherein the OMLS performs continuous online machine learning, and wherein the continuous online machine learning comprises learning or skipping learning on a per-instance or on a per-set of instances basis.

### Claim 7 (depends on 1)

The system of claim 1, wherein the OMLS is continuously operational during feature additions and feature deletions.

### Claim 8 (depends on 1)

The system of claim 1, wherein the OMLS incorporates and utilizes the one or more machine learning algorithms or models.

### Claim 9 (depends on 8)

The system of claim 8, wherein the OMLE is configured to optimize business benefit performance by optimizing at least one parameter for the one or more machine learning algorithms or models, and wherein the at least one parameter is selected according to the business benefit weighted by a determination based on at least one of topK instances, AUC (Area Under the ROC (Receiver Operating Characteristic) Curve), accuracy, weighted accuracy, or F-measure performances.

### Claim 10 (depends on 1)

The system of claim 1, wherein results of at least one of the OMLS, the OPrE, OFEE, and ORFSE are dynamically displayed for a user-customized visualization based on a role of a user in the utilization of the system.

### Claim 11 (depends on 1)

The system of claim 1, wherein at least one of the OPrE, OFEE, ORFSE, and OMLE is configured to generate an alert based on one or more detected drifts or changes in features or actual or predicted labels.

### Claim 12 (depends on 11)

The system of claim 11, wherein the system is configured for a user to perform or authorize manual or automated actions based on the one or more detected drifts or changes in the features or the actual or predicted labels, or on the alert.

### Claim 13 (depends on 12)

The system of claim 12, wherein the manual or automated actions based on the one or more detected drifts and changes or the alert include at least one of a selection of different parameters or models or automated or manual selection of weights of the models in time, or one or more business actions.

### Claim 14 (depends on 12)

The system of claim 12, wherein the manual or automated actions include one or more of saving the OMLS or one or more of components of the OMLS, or stopping or resuming training of the OMLS.

### Claim 15 (depends on 1)

The system of claim 1, wherein the system is configured for a user to perform or authorize actions that provide feedback on quality of data for a business benefit performance or a machine learning performance of the OMLS.

### Claim 16 (depends on 1)

The system of claim 1, wherein at least one of the engines in the OMLS is configured to generate an alert based on a detected excessive drift or change in relevances of all features or a user-defined or automatically-determined subset of features.

### Claim 17 (depends on 1)

The system of claim 1, wherein the ORFSE evaluates features, including with regard to a degree of relevance to a particular machine learning task.

### Claim 18 (depends on 1)

The system of claim 1, wherein the ORFSE evaluates features, including with regard to a degree of correlation to another feature.

### Claim 19 (depends on 1)

The system of claim 1, wherein the ORFSE utilizes a statistical significance test to evaluate feature relevances with respect to other actual labels or other features.

### Claim 20 (depends on 1)

The system of claim 1, wherein each of components of the OMLS are capable of being updated or skip being updated on a per-instance or on a per-set of instances basis.

### Claim 21 (depends on 1)

The system of claim 1, wherein the OMLE is capable of evaluating, weighting and selecting particular machine learning algorithms or models for optimization of operation of the OMLS.

### Claim 22 (depends on 1)

The system of claim 1, wherein the OMLE is capable of keeping and eliminating particular machine learning algorithms or models to optimize operation of the OMLS.

### Claim 23 (depends on 1)

The system of claim 1, wherein the OMLE is capable of training each of the multiple different incorporated machine learning algorithms or models according to different criteria.

### Claim 24 (depends on 23)

The system of claim 23, wherein the different criteria include different machine learning performance measures, business performance-based measure, different sets of features, outputs, or training instances.

### Claim 25 (depends on 1)

The system of claim 1, wherein the multiple machine learning algorithms or models comprise a combination of one or more decision tree-based algorithms or models, one or more clustering-based algorithms or models, one or more linear regression algorithms or models, one or more neural network algorithms or models, one or more graphical algorithms or models, or one or more mathematical expressions summarizing the model used by all of the above machine learning algorithms or models.

### Claim 26 (depends on 1)

The system of claim 1, wherein the OMLE incorporates, utilizes, evaluates and selects from the multiple machine learning algorithms or models.

### Claim 27 (depends on 1)

The system of claim 1, wherein the instance received by the OprE comprises a set of features, and wherein the OFEE is configured to enable production of additional features in addition to features in the received set of features.

### Claim 28 (depends on 1)

The system of claim 1, wherein the one or more electronic devices comprises at least one of a server, a desktop, a laptop computer, a set top box, a tablet, a cell phone, a smartphone, or a computing device capable of sending, receiving, and processing data.

### Claim 29 (depends on 1)

The system of claim 1, wherein the OFEE is configured to produce the engineered features by transforming the vectors of inputs in the set of instances that have been preprocessed by the OPrE, wherein the produced engineered features are in addition to the plurality of continuous or categorical features in the set of instances.

### Claim 30

An online machine learning method to be implemented on one or more electronic devices comprising one or more processors, the method comprising:
receiving, at an Online Preprocessing Engine (OPrE) stored on a non-transitory computer readable memory, streaming data including a set of instances, each instance of the set of instances comprising one or more vectors of inputs that include a plurality of continuous or categorical features produced externally or internally by the OMLS or by actions of one or more of users of the OMLS; and (b) discretizing features using the OPrE, wherein the OPrE is capable of imputing missing feature values, normalizing features, detecting drift or change in features, and detecting drift or change in labels for the set of instances, wherein each of the labels is an actual label or a predicted label;
producing features including engineered features, using an Online Feature Engineering Engine (OFEE) stored on a non-transitory computer readable memory;
evaluating and selecting features, using an Online Robust Feature Selection Engine (ORFSE) stored on a non-transitory computer readable memory, comprising computing a relevance value of one of the features based on a degree of correlation between a first feature and a second feature using at least one of (i) mutual information-based label relevances between features, (ii) distributional similarities between features, or (iii) similarity of change in distributions of two features to reduce correlations and redundancies between selected features, and performing a statistical significance test to determine a significance of the relevance value that is computed using at least one of (i), (ii), or (iii); and
incorporating and utilizing multiple machine learning algorithms or models utilizing features selected by the ORFSE, at an Online Machine Learning Engine (OMLE) stored on a non-transitory computer readable memory, to generate a result,
wherein each of the OMLE, the OPrE, the OFEE, and the ORFSE are continuously communicatively coupled to each other;
wherein the OPrE, which is capable of detecting the drift or change in features, is configured to detect the drift or changes in the plurality of continuous or categorical features, the discretized features, and the engineered features, and
wherein the method further comprises continuously performing, by the OMLS, operational online machine learning.

### Claim 31 (depends on 30)

The method of claim 30, further comprising utilizing an Online Explanation System (OES), stored on a non-transitory computer readable memory, to provide instance level and model level explanations to a user.

### Claim 32 (depends on 31)

The method of claim 31, wherein the OES comprises an explanation machine learning model that is trained using the vectors of inputs so that the explanation learning model is understandable by a human.

### Claim 33 (depends on 32)

The method of claim 32, wherein the explanation machine learning model also uses select engineered feature inputs that are understandable by a human.

### Claim 34 (depends on 31)

The method of claim 31, wherein the OES is configured to accommodate user preferences for explanations.

### Claim 35 (depends on 31)

The method of claim 31, wherein the OES comprises a plurality of Online Explanation Models (OEMs).

### Claim 36 (depends on 31)

The method of claim 31, wherein the OES is updated continuously.

### Claim 37 (depends on 31)

The method of claim 31, wherein the OES is separate from, but in communication with, the OMLS, and wherein the OES is updated while the OMLS is updated.

### Claim 38 (depends on 30)

The method of claim 30, further comprising utilizing an Online Human Expert Feedback System (OEFS), stored on a non-transitory computer readable memory, in obtaining, for optimization of operation of the OMLS, instance level expert feedback and model level expert feedback.

### Claim 39 (depends on 38)

The method of claim 38, further comprising utilizing human expert preferences as specified at the OEFS to control which features are used in the OMLS or how the features are used.

### Claim 40 (depends on 30)

The method of claim 30, wherein the OMLS automatically selects features, either continuously or triggered by one or more events based on machine learning performances and business benefit performances of the OMLS, wherein the one or more events comprise a change in inputs or actual or predicted outputs that exceeds a specified threshold.

### Claim 41 (depends on 30)

The method of claim 30, wherein the OMLS selects and adaptively combines one or more machine learning algorithms, models or other components based on machine learning performances and business benefit performances.

### Claim 42 (depends on 30)

The method of claim 30, wherein the OMLS implements automated feature processing methods, comprising at least one of transformation, engineering, deletion or imputation, either continuously or triggered by one or more events based on machine learning performances and business benefit performances of the OMLS, wherein the one or more events comprise a change in inputs or actual or predicted outputs that exceeds a specified threshold.

### Claim 43 (depends on 30)

The method of claim 30, wherein the step of producing features using the OFEE, comprises producing features by transforming the vectors of inputs that have been preprocessed by the OPrE into engineered feature inputs, wherein the produced features are in addition to the plurality of continuous or categorical features in the vectors of inputs.

### Claim 44

An Online Machine Learning System (OMLS) implemented on one or more electronic devices comprising one or more processors, the system comprising:
an Online Preprocessing Engine (OPrE), stored on a non-transitory computer readable memory and implemented using the one or more processors and configured to (a) receive streaming data including a set of instances, each instance of the set of instances comprising a vector of inputs that includes a plurality of continuous or categorical features produced externally or internally by the OMLS or by actions of one or more of users of the OMLS; (b) be able to discretize features; (c) be able to impute missing feature values; (d) be able to normalize features; (e) be able to detect drift or change in features; and (f) be able to detect drift or change in labels for the set of instances, wherein each of the labels is an actual label or a predicted label;
an Online Feature Engineering Engine (OFEE) stored on a non-transitory computer readable memory and implemented using the one or more processors and configured to produce features including engineered features;
an Online Robust Feature Selection Engine (ORFSE) stored on a non-transitory computer readable memory and implemented using the one or more processors and configured to evaluate and select features by computing a relevance value of one of the features based on a degree of correlation between a first feature and a second feature using at least one of (i) mutual information-based label relevances between features, (ii) distributional similarities between features, or (iii) similarity of change in distributions of two features to reduce correlations and redundancies between selected features, and performing a statistical significance test to determine a significance of the relevance value that is computed using at least one of (i), (ii), or (iii); and
an Online Machine Learning Engine (OMLE) stored on a non-transitory computer readable memory and implemented using the one or more processors and configured to incorporate and utilize one or more machine learning algorithms or models utilizing features selected by the ORFSE to generate a result, and capable of incorporating and utilizing multiple different machine learning algorithms or models;
an Online Explanation System (OES) stored on a non-transitory computer readable memory and implemented using the one or more processors and configured to provide instance level and model level explanations to a user; and
an Online Human Expert Feedback System (OEFS) stored on a non-transitory computer readable memory and implemented using the one or more processors and configured to obtain, for optimization of operation of the OMLS, instance level expert feedback and model level expert feedback;
wherein each of the OMLE, the OPrE, the OFEE, the ORFSE, the OES and the OEFS are continuously communicatively coupled to each other and capable of continuous updating;
wherein the OPrE, which is configured to be able to detect the drift or change in features, is configured to detect the drift or changes in the plurality of continuous or categorical features, the discretized features, and the engineered features, and
wherein the OMLS is configured to perform continuous online machine learning comprising per-instance learning.

### Claim 45

A non-transitory computer readable medium or media containing instructions for executing a method for performing online machine learning using one or more electronic devices comprising one or more processors, the method comprising:
(a) receiving, at an Online Preprocessing Engine (OPrE), streaming data including a set of instances, each instance of the set of instances comprising a vector of inputs that includes a plurality of continuous or categorical features produced externally or internally by the OMLS or by actions of one or more of users of the OMLS; and (b) discretizing features using the OPrE, wherein the OPrE is capable of imputing missing feature values, normalizing features, and detecting drift or change in features and labels for the set of instances, wherein each of the labels is an actual label or a predicted label;
producing features including engineered features, using an Online Feature Engineering Engine (OFEE);
evaluating and selecting features, using an Online Robust Feature Selection Engine (ORFSE), comprising computing a relevance value of one of the features based on a degree of correlation between a first feature and a second feature using at least one of (i) mutual information-based label relevances between features, (ii) distributional similarities between features, or (iii) similarity of change in distributions of two features to reduce correlations and redundancies between selected features, and performing a statistical significance test to determine a significance of the relevance value that is computed using at least one of (i), (ii), or (iii); and
incorporating and utilizing multiple machine learning algorithms or models utilizing features, selected by the ORFSE, at an Online Machine Learning Engine (OMLE), to generate a result;
wherein each of the OMLE, the OPrE, the OFEE and the ORFSE are continuously communicatively coupled to each other;
wherein the OPrE, which is capable of detecting the drift or change in features, is capable of detecting the drift or changes in the plurality of continuous or categorical features, the discretized features, and the engineered features, and
wherein the method further comprises continuously operational online machine learning by the OMLS.