Systems and methods for log analysis using large language models and quantum computational techniques
AssigneeJPMorgan Chase Bank, N.A.
InventorSanjay GOEL
Life of this patent
Estimating the term…
Abstract
Embodiments may provide a quantum-assisted system for analyzing log data and clustering errors based on frequency and semantic similarity. The system processes log data from CSV files, vectorizes error messages using SpaCy, stores embeddings in Chroma Database (optional), and applies quantum computation to optimize clustering with centroids. By identifying the top errors within each cluster, the system offers advanced insights for troubleshooting. An orchestration subsystem may automatically execute an action determined based on the advanced insights in the target environment.
Claims20
A computer-implemented method for analyzing log data, the method comprising:
- ingesting, by an application executed by one or more processors, log entries comprising unstructured text from one or more computing resources;
- preprocessing, by the application in communication with a natural language processing pipeline that includes a spaCy large language model, the log entries to generate normalized text, wherein preprocessing comprises tokenizing the log entries, performing named entity recognition, and removing stop-words;
- vectorizing, by the application, the normalized text to generate feature vectors representing respective ones of the log entries;
- generating, by the one or more processors, a quantum-derived random value by executing a quantum circuit on a quantum computer or a quantum computer simulator;
- initializing, by the application, a plurality of centroids for clustering based on (i) randomly selected log-entry indices and (ii) an adjustment to at least one of the randomly selected log-entry indices responsive to the quantum-derived random value;
- clustering, by the application, the feature vectors into clusters using an unsupervised clustering algorithm that uses the plurality of centroids as initial centroids;
- generating, by the application, a cluster summary for at least one cluster based on log entries assigned to the at least one cluster;
- and
- outputting, by the application, at least one of (i) a table identifying a plurality of frequent errors within the clusters or (ii) a visualization of cluster membership.
A system for analyzing log data, comprising:
- one or more processors;
- and
- one or more non-transitory storage devices storing instructions that, when executed by the one or more processors, cause the system to:
- ingest log entries comprising unstructured text;
- preprocess the log entries using a natural language processing pipeline that includes a spaCy large language model to generate normalized text, wherein preprocessing comprises tokenization, named entity recognition, and stop-word removal;
- vectorize the normalized text to generate feature vectors;
- obtain a quantum-derived random value by executing a quantum circuit on a quantum computer or a quantum computer simulator;
- initialize centroids for clustering based on randomly selected indices and an adjustment responsive to the quantum-derived random value;
- cluster the feature vectors into clusters using an unsupervised clustering algorithm initialized with the centroids;
- and
- generate a summary for at least one cluster based on log entries assigned to the at least one cluster;
- and provide an output comprising at least one of (i) a table of frequent errors by cluster or (ii) a visualization of clusters.
A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause a computing system to perform operations comprising:
- receiving log entries comprising unstructured text;
- preprocessing the log entries using a natural language processing pipeline that includes a spaCy large language model to generate normalized text;
- generating feature vectors from the normalized text;
- generating a quantum-derived random value by executing a quantum circuit using a quantum computer or a quantum computer simulator;
- initializing clustering centroids based on a random selection of feature-vector indices and a quantum-driven adjustment to at least one selected index;
- clustering the feature vectors into clusters using the initialized clustering centroids;
- and
- producing cluster-level outputs that include at least one of (i) a summary of a cluster or (ii) frequent-error statistics for the cluster;
- and causing display of the cluster-level outputs.
Dependencies are inferred from the claim text.