Method for scalable mining of temporally correlated events
AssigneeInternational Business Machines Corporation
InventorsVinh Tuan Thai, Luke Taher, Mark O'Riordan
Life of this patent
Estimating the term…
Abstract
Scalable mining method, computer system and computer program product for temporally correlated events that works with input data that is non-transactional in nature and detects high correlation relationships between events with low support thresholds. The scalable mining method manages a large scale of input data by executing the mining process in a distributed manner in a clustered cloud environment via nodes.
Claims20
A method of scalable mining of temporally correlated events comprising the steps of:
- a computer recording event occurrences as bitmaps at least at a fine grained interval and a coarse grained interval into a bitmap data structure;
- the computer filtering out events occurrences which occur above and below a minimum and maximum designated thresholds relative to the fine grained interval and the coarse grained interval;
- the computer identifying events occurrences having a same event occurrence bitmap at the fine grained interval and the coarse grained interval and placing the same event occurrence bitmaps into groups;
- the computer removing duplicate event occurrence bitmaps at the fine grained intervals within the groups;
- the computer creating transactions based on the groups;
- for each transaction created, applying optimized parallel frequent pattern growth to a plurality of other distributed computers to produce result sets;
- the computer merging all result sets from the plurality of distributed computers;
- the computer applying rule filtering and outputting remaining groups of events;
- and
- the computer representing the remaining groups as a distributed graph to output merged groups of temporally correlated events.
A computer program product for scalable mining of temporally correlated events using a computer comprising at least one processor, one or more memories, one or more computer readable storage media, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by the computer to perform a method comprising:
- recording, by the computer event occurrences as bitmaps at least at a fine grained interval and a coarse grained interval into a bitmap data structure;
- filtering out, by the computer, events occurrences which occur above and below a minimum and maximum designated thresholds relative to the fine grained interval and the coarse grained interval;
- identifying, by the computer, events occurrences having a same event occurrence bitmap at the fine grained interval and the coarse grained interval and placing the same event occurrence bitmaps into groups;
- removing, by the computer, duplicate event occurrence bitmaps at the fine grained intervals within the groups;
- creating, by the computer, transactions based on the groups;
- for each transaction created, applying, by the computer, optimized parallel frequent pattern growth to a plurality of other distributed computers to produce result sets;
- merging, by the computer, all result sets from the plurality of distributed computers;
- applying, by the computer, rule filtering and outputting remaining groups of events;
- and
- representing, by the computer, the remaining groups of events as a distributed graph to output merged groups of temporally correlated events.
A computer system for scalable mining of temporally correlated events comprising a computer comprising at least one processor, one or more memories, one or more computer readable storage media having program instructions executable by the computer to perform the program instructions comprising:
- recording, by the computer event occurrences as bitmaps at least at a fine grained interval and a coarse grained interval into a bitmap data structure;
- filtering out, by the computer, events occurrences which occur above and below a minimum and maximum designated thresholds relative to the fine grained interval and the coarse grained interval;
- identifying, by the computer, events occurrences having a same event occurrence bitmap at the fine grained interval and the coarse grained interval and placing the same event occurrence bitmaps into groups;
- removing, by the computer, duplicate event occurrence bitmaps at the fine grained intervals within the groups;
- creating, by the computer, transactions based on the groups;
- for each transaction created, applying, by the computer, optimized parallel frequent pattern growth to a plurality of other distributed computers to produce result sets;
- merging, by the computer, all result sets from the plurality of distributed computers;
- applying, by the computer, rule filtering and outputting remaining groups of events;
- and
- representing, by the computer, the remaining groups of events as a distributed graph to output merged groups of temporally correlated events.
Dependencies are inferred from the claim text.