Scalable window aggregator
OwnerMangesh Bendre, Fei Wang +1inventors
InventorsMangesh Bendre, Fei Wang, Hao Yang
Life of this patent
Estimating the term…
Abstract
A method performed by a computer is disclosed. The method includes receiving a dataset including one or more data fields. The computer can then receive instructions to compute an aggregate feature of the dataset. The computer can then determine an operator and a window based on the aggregate feature to be computed. The method may then include the computer processing the dataset. The compute may then apply the operator with the window to the processed dataset to compute the aggregate feature of the dataset.
Claims20
A method of processing a dataset by a computer, the method comprising:
- storing the dataset comprising N data measurement vectors, each data measurement vector comprising a time field and a data field, wherein N is at least 100,000, wherein the data field of each data measurement vector comprises a measured value of a property of a network system for a corresponding time value;
- receiving a command to compute an aggregate feature vector of an aggregate feature of the dataset, wherein the aggregate feature vector includes a respective aggregate value of the data field for each of a plurality of sliding time windows spanning the dataset;
- determining an operator for determining the respective aggregate values and a window size for the aggregate feature to be computed;
- for a first time window, determining with the operator a first aggregate value using the measured values of first data measurement vectors having the time field within the first time window;
- advancing a window pointer to define a second time window, wherein the second time window overlaps with the first time window, and wherein the second time window includes second data measurement vectors;
- identifying a first portion of the first data measurement vectors that are not within the second time window;
- identifying a second portion of the second data measurement vectors that are not within the first time window;
- determining, using the operator, a second aggregate value by:
- removing the measured value(s) for the first portion of the first data measurement vectors from the first aggregate value;
- and
- including the measured value(s) for the second portion of the second data measurement vectors within the first aggregate value;
- and
- repeating advancing of the window pointer for successive time windows and repeating identification of data measurement vectors to remove and to include, for the operator to determine successive aggregate values for the successive time windows of the plurality of sliding time windows.
A computer comprising:
- a processor;
- and
- a non-transitory computer readable medium comprising instructions executable by the processor to perform a method of processing a dataset by a computer, the method comprising:
- storing the dataset comprising N data measurement vectors, each data measurement vector comprising a time field and a data field, wherein N is at least 100,000, wherein the data field of each data measurement vector comprises a measured value of a property of a network system for a corresponding time value;
- receiving a command to compute an aggregate feature vector of an aggregate feature of the dataset, wherein the aggregate feature vector includes a respective aggregate value of the data field for each of a plurality of sliding time windows spanning the dataset;
- determining an operator for determining the respective aggregate values and a window size for the aggregate feature to be computed;
- for a first time window, determining with the operator a first aggregate value using the measured values of first data measurement vectors having the time field within the first time window;
- advancing a window pointer to define a second time window, wherein the second time window overlaps with the first time window, and wherein the second time window includes second data measurement vectors;
- identifying a first portion of the first data measurement vectors that are not within the second time window;
- identifying a second portion of the second data measurement vectors that are not within the first time window;
- determining, using the operator, a second aggregate value by:
- removing the measured value(s) for the first portion of the first data measurement vectors from the first aggregate value;
- and
- including the measured value(s) for the second portion of the second data measurement vectors within the first aggregate value;
- and
- repeating advancing of the window pointer for successive time windows and repeating identification of data measurement vectors to remove and to include, for the operator to determine successive aggregate values for the successive time windows of the plurality of sliding time windows.
Dependencies are inferred from the claim text.