# US 2022/0036201 A1 · Real drift detector on partial labeled data in data streams

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0036201 A1
- Document ID: US20220036201A1
- Status: Published application
- Kind: A1
- Application number: 16/945,876
- Filed: 2020-08-02
- Published: 2022-02-03
- Earliest claimed: 2020-08-02
- Assignees: Actimize Ltd.
- Applicants: Actimize LTD.
- Inventors: Ganir TAMIR; Danny BUTVINIK; Yoav AVNEON
- Web page: https://patentgrep.com/patent/US20220036201A1

## CPC

- G06N 5/01: Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
- G06F 18/10: Pre-processing; Data cleansing
- G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06N 20/10: using kernel methods, e.g. support vector machines [SVM]
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A computerized-method for real-time detection of real concept drift in predictive machine learning models, by processing high-speed streaming data. The computerized-method includes: receiving a real-time data stream having labeled and unlabeled instances. Obtaining a window of ‘n’ instances having a portion of the ‘n’ instances as reliable labels. Computing posterior distribution of the reliable labels; and operating a Drift-Detection (DD) module. The DD module is configured to: operate a kernel density estimation on the computed posterior distribution for sensitivity control of the DD module; operate an error rate function on the estimated kernel density to yield an error value; and train an incremental estimator module, according to the kernel density estimation. When the error value is not above a preconfigured drift threshold repeating operations (i) through (iii), else when the error value is above the preconfigured drift threshold, at least one concept drift related action takes place.

## Claims

### Claim 1

A computerized-method for real-time detection of real concept drift in predictive machine learning models, by processing high-speed streaming data, said computerized-method comprising:
in a computerized-system comprising a processor and a memory, receiving by the processor, a real-time data stream having labeled and unlabeled instances,
every preconfigured period of time, performing by the processor:
(i) obtaining a window of ‘n’ instances from the data stream having a portion of the ‘n’ instances as reliable labels;
(ii) computing posterior distribution of the reliable labels of the portion of the ‘n’ instances; and
(iii) operating a Drift-Detection (DD) module, said DD module is configured to:
a. operating a kernel density estimation on the computed posterior distribution for sensitivity control of the DD module;
b. operating an error rate function on the estimated kernel density to yield an error value; and
c. training an incremental estimator module, according to the kernel density estimation;
when the error value is not above a preconfigured drift threshold repeat operations (i) through (iii),
else when the error value is above the preconfigured drift threshold, at least one real concept drift related action takes place.

### Claim 2 (depends on 1)

The computerized-method of claim 1, wherein after obtaining a window of ‘n’ instances from the data stream, the processor is further configured to:
counting the labeled instances in the ‘n’ instances;
multiplying a labeling cost by the counted labeled instances to yield a total-cost;
when the total-cost is not above a preconfigured labeling budget:
operating a Knowledge Discovery (KD) module to obtain reliable labels of the portion of the ‘n’ instances by applying one or more machine learning models; and
performing operations (ii) through (iii).

### Claim 3 (depends on 2)

The computerized-method of claim 2, before the performing of operations (ii) through (iii), further comprising:
initiating and training of a static estimator, according to the obtained reliable labels to provide the DD module a posterior distribution.

### Claim 4 (depends on 1)

The computerized-method of claim 1, wherein the reliable labels of the portion of the ‘n’ instances are provided by an end-user before the obtaining of a window of ‘n’ instances from the data stream.

### Claim 5 (depends on 1)

The computerized-method of claim 1, wherein when the error value is not above a predefined warning threshold, training the incremental estimator module with the obtained reliable labels of the portion of the ‘n’ instances.

### Claim 6 (depends on 1)

The computerized-method of claim 1, wherein the error rate function is a Gauss error function.

### Claim 7 (depends on 1)

The computerized-method of claim 1, wherein the posterior distribution is a sum of a prior distribution and a result of a likelihood function.

### Claim 8 (depends on 1)

The computerized-method of claim 1, wherein the operating of the incremental estimator module comprises operating an incremental decision tree learner and training the incremental estimator module, according to the kernel density estimation.

### Claim 9 (depends on 8)

The computerized-method of claim 8, wherein the incremental decision tree learner is an Hoeffding tree.

### Claim 10 (depends on 1)

The computerized-method of claim 1, wherein the at least one real concept drift related action which takes place when the predictive machine learning models are operating in batch mode, is an alert to the predictive machine learning models to go offline and re-train for later deployment.

### Claim 11 (depends on 1)

The computerized-method of claim 1, wherein the at least one real concept drift related action which takes place when the predictive machine learning models are operating online, is an alert to the predictive machine learning models to adjust to changing trend of statistics.

### Claim 12 (depends on 3)

The computerized-method of claim 3, wherein the DD module receives a posterior distribution from the incremental estimator and the static estimator.