System and method with machine learning framework for system forensics

Abstract

A computer-implemented method and system relate to performing a task in a time series domain using different modalities of time series information. Time series data is received and includes numerical data for a set of channels. A digital image is generated and displays a set of graphical representations of the time series data. A tokenizer generates time series embeddings using the time series data in a channel-independent manner to preserve local semantic information. An image encoder generates image embeddings using the digital image, thereby capturing global context and cross-channel dependencies of the time series data. A pretrained large language model (LLM) generates semantic embeddings based on at least the time series embeddings and the image embeddings. At least one task head performs the task and generates output data for the task using the semantic embeddings. The task may include classification, anomaly detection, forecasting, imputation, clustering, etc.

Dependencies are inferred from the claim text.