Rationalization of data used in model of time varying event behavior
AssigneeBritish Telecommunications public limited company
InventorsMichaela M Black, Raymond J Hickey
Life of this patent
Estimating the term…
Abstract
A method for rationalization of data used to model a time-variant behavior provides advantages in that storage requirements for such data are reduced and accuracy of detection of events in the behavior is increased. The method uses labels added to training data to indicate whether that data relates to recent events or not. A classifier is generated from the labelled training data. By removing old data which the classifier would classify differently were the old data re-labelled as new, a selective purging of the old training data takes place each time new training data becomes available. The method is especially useful in detecting fraudulent use of, or faults in, a communications network.
Claims15
A computer-implemented method for operating data processing apparatus to rationalize stored training data, said training data being suitable for generating a model of a time-valiant behavior wherein said training data includes an extant set of training examples, each member of said extant set providing values for a newness parameter and one or more other parameters, the value of said newness parameter indicating that the event to which that training example relates occurred before a predetermined time;
- said method comprising:
- (a) gathering an update set of training examples, each member of said update set providing values for said newness parameter and said one or more other parameters, the value of said newness parameter indicating that the event to which that training example relates occurred after said predetermined time;
- (b) analyzing said extant set and said update set to generate a classifier which classifies training examples from both said extant set and said update set, the classification of the training examples being a function of a plurality of parameters which include said newness parameter;
- and
- (c) forming a surviving set of training examples by selecting, on the basis of the generated classifier, a surviving set of training examples from said extant set.
Apparatus comprising an input device, memory, an output device, and a data processing unit, each of said devices being connected in operation to said data processing unit, wherein said memory stores:
- a) extant training data input code executable by said data processing unit to cause extant training data representations to be also represented at predetermined locations in said memory, said extant training data comprising a plurality of extant training examples, each extant training example providing values for a newness parameter and a set of one or more other parameters, the value of said newness parameter indicating that said event occurred before a predetermined time;
- b) update training data input code executable by said data processing unit to cause update training data representations presented to said input device to be also represented at further predetermined locations in said memory, said update training data comprising a plurality of update training examples, each update training example providing values for a newness parameter and a set of one or more other parameters, the value of said newness parameter indicating that said event occurred after a predetermined time;
- c) a classifier generation code executable by said data processing unit to generate a classifier which is able to classify training examples from both said extant set and said update set, the classification of the training examples being dependent on the value of one or more of said one or more other parameters and said newness parameter;
- and
- d) training data purging code executable by said data processing unit to remove from said extant training data at least some of those examples which the generated classifier would classify differently were the value of said newness parameter to be changed to indicate that the event to which the example relates occurred after said predetermined time.
A computer-implemented method of operating a data processing apparatus to process classifier training data to provide purged classifier training data, said data processing apparatus comprising an input device, an output device, memory and a data processing unit, said method comprising:
- storing training data in said memory, said training data comprising:
- (a) an extant set of training examples, each member of said extant set providing values for a newness parameter and one or more other parameters, the value of said newness parameter indicating that the event to which that training example relates occurred before a predetermined time;
- and
- (b) an update set of training examples, each member of said update set providing values for said newness parameter and said one or more other parameters, the value of said newness parameter indicating that the event to which that training example relates occurred after said predetermined time;
- operating said data processing unit to process said training data to generate a classifier which is able to classify training examples from both said extant set and said update set, the classification of the training examples being dependent on the value of one or more of said one or more other parameters and said newness parameter;
- and
- operating said data processing unit to remove from said extant set those training examples which the generated classifier would classify differently were the value of said newness parameter to be changed to indicate that the event to which the example relates occurred after said predetermined time.
Dependencies are inferred from the claim text.