Systems and methods for detecting and grouping anomalies in data
AssigneeCable Television Laboratories, Inc.
InventorsJingjie Zhu, Karthik Sundaresan
Life of this patent
Estimating the term…
Abstract
The present disclosure generally relates to apparatus, software and methods for detecting anomalous elements in data. For example, the data can be any time series, such as but not limited to radio frequency data, temperature data, stock data, or production data. Each type of data may be susceptible to repeating phenomena that produce recognizable features of anomalous elements. In some embodiments, the features can be characterized as known patterns and used to train a machine learning model via supervised learning to recognize those features in a new data series.
Claims18
A method for detecting anomalous elements in data comprising:
- generating a data series from a plurality of signals received from one or more cable modems operating on a communication network;
- detrending the data series to obtain an un-tilted data series;
- designating a number of elements within the un-tilted data series as a sliding window;
- moving the sliding window in increments across the un-tilted data series;
- calculating a median and a standard deviation for the sliding window at each increment;
- when the standard deviation for the sliding window at one of the increments is greater than a threshold (SD threshold), identifying anomalous elements within the sliding window;
- replacing the anomalous elements with the median of the sliding window to create a modified sliding window at the one increment;
- calculating a new median and a new standard deviation for the modified sliding window;
- identifying new anomalous elements within the modified sliding window when the new standard deviation is greater than a new SD threshold;
- scoring the data series based on a quantity of the new anomalous elements, a magnitude of the new anomalous elements, or both a quantity and a magnitude of the new anomalous elements;
- when the score is greater than or equal to a specified number, identifying which of the one or more cable modems from which the data series is obtained is an impaired cable modem;
- and
- transmitting signals to/from the impaired cable modem outside frequencies associated with the new anomalous elements.
A non-transitory computer-readable medium having a plurality of non-transitory instructions executable with a processor for detecting anomalous elements in data, the plurality of non-transitory instructions being executable for:
- generating a data series from a plurality of signals received from one or more cable modems operating on a communication network;
- detrending the data series to obtain an un-tilted data series;
- designating a number of elements within the un-tilted data series as a sliding window;
- moving the sliding window in increments across the un-tilted data series;
- calculating a median and a standard deviation for the sliding window at each increment;
- when the standard deviation for the sliding window at one of the increments is greater than a threshold (SD threshold), identifying anomalous elements within the sliding window;
- replacing the anomalous elements with the median of the sliding window to obtain a modified sliding window at the one increment;
- calculating a new median and a new standard deviation for the modified sliding window;
- identifying new anomalous elements within the modified sliding window when the new standard deviation is greater than a new SD threshold;
- scoring the data series based on a quantity of the new anomalous elements, a magnitude of the new anomalous elements, or both a quantity and a magnitude of the new anomalous elements;
- when the score is greater than or equal to a specified number, identifying which of the one or more cable modems from which the data series is obtained is an impaired cable modem;
- and
- transmitting signals to/from the impaired cable modem outside frequencies associated with the new anomalous elements.
Dependencies are inferred from the claim text.