Time-series data forecasting via multi-modal augmentation and fusion

A system for time-series forecasting via multi-modal augmentation and fusion, comprising:

• a memory device;
• and
• one or more processor devices operatively coupled with the memory device to:
• decompose time-series data and modality data into seasonal and trend representations with trend-seasonal decomposition;
• concatenate, using an encoder transformer model, time-series data embeddings and modality data embeddings from the seasonal representations and the trend representations to obtain crossed representations;
• process, using the encoder transformer model, the modality data embeddings and the time-series data embeddings separately to obtain singular representations;
• augment the crossed representations and the singular representations through joint trend-seasonal decomposition to obtain augmented seasonal data and augmented trend data;
• fuse, using a decoder, augmented seasonal data and augmented trend data to obtain fused augmented data;
• and
• perform corrective action to correct predicted future events using a system with a prediction model trained with the fused augmented data.

A non-transitory computer program product comprising a computer-readable storage medium including program code for time-series forecasting via multi-modal augmentation and fusion, wherein the program code when executed on a computer causes the computer to:

• decompose time-series data and modality data into seasonal and trend representations with trend-seasonal decomposition;
• concatenate, using an encoder transformer model, time-series data embeddings and modality data embeddings from the seasonal representations and the trend representations to obtain crossed representations;
• process, using the encoder transformer model, the modality data embeddings and the time-series data embeddings separately to obtain singular representations;
• augment the crossed representations and the singular representations through joint trend-seasonal decomposition to obtain augmented seasonal data and augmented trend data;
• fuse, using a decoder, augmented seasonal data and augmented trend data to obtain fused augmented data;
• and
• perform corrective action to correct predicted future events using a system with a prediction model trained with the fused augmented data.