US 2023/0401512 A1 · MITIGATING TEMPORAL GENERALIZATION FOR A MACHINE LEARNING MODEL

Abstract

Mitigation of temporal generalization losses a target machine learning model is disclosed. Mitigation can be based on identifying, removing, modifying, transforming, etc., features, explanatory variables, models, etc., that can have an unstable relationship with a target outcome over time. Implementation of a more stable representation can be initiated. Temporal stability measures (TSMs) for one or more model feature(s) can be determined based on one or more variable performance metrics (VPMs). A group of one or more VPMs can be selected based on features of a model in either a development or production environment. Model feature modification can be recommended based on a TSM, which can prune a feature, transform a feature, add a feature, etc. Temporal stability information can be presented, e.g., via a dashboard-type user interface. Models can be updated based on mutations of a model comprising a feature modification(s), including competitive champion/challenger model updating.

Claims20

Dependent claims

A device, comprising:

a processor;
and
a memory that stores executable instructions that, when executed by the processor, facilitate performance of operations, comprising:
determining, in response to receiving incoming model data for a model, a variable performance metric corresponding to a feature of the model;
determining, based on a behavior of the variable performance metric as a function of time, a temporal stability measurement;
determining, based on the temporal stability measurement, a feature-target relationship for the model;
and
enabling access to information about the feature-target relationship.

5 more elements

A method, comprising:

selecting, by a processor of a system, a group of variable performance metrics corresponding to features of a first trained model, wherein the selecting employs a second trained model to indicate features of the first trained model that are predicted to have more influence on a performance of the first trained model;
determining, by the processor, a group of temporal stability measurements based on analysis of the first trained model relative to variable performance metric of the group of variable performance metrics;
determining, by the processor, a feature-target relationship for the first trained model based on the group of temporal stability measurements;
and
presenting, by the processor, information about the feature-target relationship via a user interface.

2 more elements

A non-transitory machine-readable storage medium, comprising executable instructions that, when executed by a processor, facilitate performance of operations, comprising:

selecting a group of variable performance metrics corresponding to features of a first trained model, wherein the selecting employs a second trained model to indicate features of the first trained model that are predicted to have a greater effect on a performance of the first trained model;
determining a group of temporal stability measurements based on analysis of the first trained model relative to variable performance metric of the group of variable performance metrics;
determining a feature-target relationship for the first trained model based on the group of temporal stability measurements;
determining a third trained model based on the first trained model and a mutation selected from a group of mutations, wherein the mutation is selected based on a temporal stability measurement of the group of temporal stability measurements;
and
presenting, by the processor, information about the feature-target relationship and the third trained model via a user interface.

3 more elements

Dependencies are inferred from the claim text.

Mitigating temporal generalization for a machine learning model

Life of this patent

Mitigating temporal generalization for a machine learning model

Life of this patent

Abstract

Claims20