# US 12,602,611 B2 · Dynamic updating of machine learning models

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,602,611 B2
- Document ID: US12602611B2
- Status: Granted
- Kind: B2
- Application number: 17/504,418
- Filed: 2021-10-18
- Published: 2022-02-03
- Granted: 2026-04-14
- Earliest claimed: 2017-03-31
- Assignees: AT&T Intellectual Property I, L.P.
- Applicants: AT&T Intellectual Property I, L.P.
- Inventors: Raghuraman Gopalan
- Web page: https://patentgrep.com/patent/US12602611B2

## CPC

- G06N 20/00: Machine learning
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A method may include a processor training a machine learning model with a training data set, computing a data distribution of the training data set, processing a stream of new data to determine a likelihood of the new data from the data distribution that is computed, incrementing a counter when the likelihood of the new data is less than a first threshold, and retraining the machine learning model when the counter exceeds a second threshold.

## Claims

### Claim 1

A device comprising:
a processor; and
a computer-readable medium storing instructions which, when executed by the processor, cause the processor to perform operations, the operations comprising:
training a machine learning model with a training data set;
computing a data distribution of the training data set;
processing a stream of new data to determine a probability of the new data from a data distribution of the new data and the data distribution of the training data set that is computed;
incrementing a counter when the probability of the new data is less than a first threshold;
determining, in response to the counter exceeding a second threshold, a future time at which to retrain the machine learning model;
retraining the machine learning model at the future time that is determined;
calculating an optimization function that identifies a plurality of relevance scores of a plurality of features of the machine learning model with respect to a service of the machine learning model; and
transmitting an instruction to at least one network element to decrease at least one of: a data sampling rate or a rate of data collection with respect to at least a first feature of the plurality of features with at least a first relevance score of the plurality of relevance scores that is below a third threshold.

### Claim 2 (depends on 1)

The device of claim 1, wherein the data distribution of the training data set is calculated as a parametric distribution or a non-parametric distribution.

### Claim 3 (depends on 1)

The device of claim 1, wherein the data distribution of the training data set comprises a Gaussian distribution, a Laplacian distribution, or a Parzen window.

### Claim 4 (depends on 1)

The device of claim 1, wherein the machine learning model comprises a classifier.

### Claim 5 (depends on 1)

The device of claim 1, wherein the training data set and the stream of new data comprise network traffic data.

### Claim 6 (depends on 1)

The device of claim 1, wherein the training data set and the stream of new data comprise network call center data.

### Claim 7 (depends on 1)

The device of claim 1, wherein the operations further comprise:
applying the machine learning model to the stream of new data to generate an inference.

### Claim 8 (depends on 7)

The device of claim 7, wherein the inference comprises a classification.

### Claim 9 (depends on 1)

The device of claim 1, wherein the operations further comprise:
providing an ordered list of the plurality of features of the machine learning model organized by the plurality of relevance scores.

### Claim 10 (depends on 1)

The device of claim 1, wherein the operations further comprise:
sending a second instruction to the at least one network element to increase a rate of data collection with respect to at least a second feature of the plurality of features with at least a second relevance score of the plurality of relevance scores that exceeds the third threshold.

### Claim 11 (depends on 10)

The device of claim 10, wherein additional data that is collected by the network element in accordance with the second instruction is used to retrain the machine learning model.

### Claim 12 (depends on 1)

The device of claim 1, wherein the operations further comprise:
sending a second instruction to the at least one network element to increase a data sampling rate with respect to at least a second feature of the plurality of features with at least a second relevance score of the plurality of relevance scores that exceeds the third threshold.

### Claim 13 (depends on 12)

The device of claim 12, wherein additional data that is collected by the network element in accordance with the second instruction is used to retrain the machine learning model.

### Claim 14 (depends on 1)

The device of claim 1, wherein the instruction comprises an instruction to the at least one network element to decrease the data sampling rate with respect to the at least the first feature of the plurality of features with the at least the first relevance score of the plurality of relevance scores that is below the third threshold.

### Claim 15 (depends on 1)

The device of claim 1, wherein the instruction comprises an instruction to the at least one network element to decrease the rate of data collection with respect to the at least the first feature of the plurality of features with the at least the first relevance score of the plurality of relevance scores that is below the third threshold.

### Claim 16 (depends on 1)

The device of claim 1, wherein the probability comprises a distance between the data distribution of the training data set and the data distribution of the stream of new data.

### Claim 17 (depends on 1)

The device of claim 1, wherein the probability comprises an integral of a difference between the data distribution of the training data set and the data distribution of the stream of new data.

### Claim 18

A method comprising:
training, by a processor, a machine learning model with a training data set;
computing, by the processor, a data distribution of the training data set;
processing, by the processor, a stream of new data to determine a probability of the new data from a data distribution of the new data and the data distribution of the training data set that is computed;
incrementing, by the processor, a counter when the probability of the new data is less than a first threshold;
determining, by the processor in response to the counter exceeding a second threshold, a future time at which to retrain the machine learning model;
retraining, by the processor, the machine learning model at the future time that is determined;
calculating, by the processor, an optimization function that identifies a plurality of relevance scores of a plurality of features of the machine learning model with respect to a service of the machine learning model; and
transmitting, by the processor to at least one network element, an instruction to decrease at least one of: a data sampling rate or a rate of data collection with respect to at least a first feature of the plurality of features with at least a first relevance score being of the plurality of relevance scores that is below a third threshold.

### Claim 19

A non-transitory computer-readable storage medium storing instructions which, when executed by a processor, cause the processor to perform operations, the operations comprising:
training a machine learning model with a training data set;
computing a data distribution of the training data set;
processing a stream of new data to determine a probability of the new data from a data distribution of the new data and the data distribution of the training data set that is computed;
incrementing a counter when the probability of the new data is less than a first threshold;
determining, in response to the counter exceeding a second threshold, a future time at which to retrain the machine learning model;
retraining the machine learning model at the future time that is determined;
calculating an optimization function that identifies a plurality of relevance scores of a plurality of features of the machine learning model with respect to a service of the machine learning model; and
transmitting an instruction to at least one network element to decrease at least one of: a data sampling rate or a rate of data collection with respect to at least a first feature of the plurality of features with at least a first relevance score of the plurality of relevance scores that is below a third threshold.

### Claim 20 (depends on 19)

The non-transitory computer-readable storage medium of claim 19, wherein the data distribution of the training data set is calculated as a parametric distribution or a non-parametric distribution.