Data selection and sampling system for log parsing and anomaly detection in cloud microservices
AssigneeInternational Business Machines Corporation
InventorsXiaotong Liu, Jiayun Zhao, Anbang Xu, Rama Kalyani T. Akkiraju
Life of this patent
Estimating the term…
Abstract
A method for log data sampling is disclosed. The method includes receiving logs of a computer system. A log comprises information regarding an operation of the computer system. The method also includes determining a sample of the logs by applying a set of sampling methods to the logs. The method further includes providing the sample of the logs as an input to an anomaly detection model for the computer system. The anomaly detection model identifies a fault in the operation of the computer system based on the input.
Claims20
A method comprising:
- receiving logs of a computer system, wherein a log comprises information regarding an operation of the computer system;
- determining a sample of the logs by applying a set of sampling methods to the logs to:
- determine that the set of sampling methods includes a first sampling method and a second sampling method;
- determine a first sub-sample by applying the first sampling method to the logs;
- determine a second sub-sample by applying the second sampling method to the rest of the logs other than the first sub-sample;
- and
- determine the first sub-sample and the second sub-sample as the sample of the logs;
- and
- providing the sample of the logs as an input to an anomaly detection model for the computer system, wherein the anomaly detection model identifies a fault in the operation of the computer system based on the input.
(canceled)
A system comprising:
- a memory having instructions;
- and
- at least one processor in communication with the memory, wherein the at least one processor is configured to:
- receive logs of a computer system, wherein a log comprises information regarding an operation of the computer system, and wherein the computer system supports microservices;
- determine a sample of the logs by applying a set of sampling methods to the logs to:
- determine that the set of sampling methods includes a first sampling method and a second sampling method;
- determine a first sub-sample by applying the first sampling method to the logs;
- determine a second sub-sample by applying the second sampling method to the rest of the logs other than the first sub-sample;
- and
- determine the first sub-sample and the second sub-sample as the sample of the logs;
- and
- provide the sample of the logs as an input to an anomaly detection model for the computer system, wherein the anomaly detection model identifies a fault in the operation of the computer system based on the input.
(canceled)
A computer program product comprising:
- a computer readable storage medium having program instructions that are executable by at least one processor to cause the at least one processor to:
- receive logs of a computer system, wherein a log comprises information regarding an operation of the computer system;
- determine a sample of the logs by applying a set of sampling methods to the logs to:
- determine that the set of sampling methods includes a first sampling method and a second sampling method;
- determine a first sub-sample by applying the first sampling method to the logs;
- determine a second sub-sample by applying the second sampling method to the rest of the logs other than the first sub-sample;
- and
- the first sub-sample and the second sub-sample as the sample of the logs;
- and
- provide the sample of the logs as an input to an anomaly detection model for the computer system, wherein the anomaly detection model identifies a fault in the operation of the computer system based on the input.
(canceled)
Dependencies are inferred from the claim text.