Embedding multi-modal time series and text data
AssigneeNEC Laboratories America, Inc.
InventorsYuncong Chen, Dongjin Song, Cristian Lumezanu, Haifeng Chen, Takehiko Mizoguchi, Xuchao Zhang
Life of this patent
Estimating the term…
Abstract
Methods and systems of training and using a neural network model include training a time series embedding model and a text embedding model with unsupervised clustering to translate time series and text, respectively, to a shared latent space. The time series embedding model and the text embedding model are further trained using semi-supervised clustering that samples training data pairs of time series information and associated text for annotation.
Claims20
A method of training a neural network, comprising:
- training, using a hardware processor, a time series embedding model and a text embedding model with unsupervised clustering to translate time series and text, respectively, to a shared latent space;
- and
- training the time series embedding model and the text embedding model further using semi-supervised clustering that samples training data pairs of time series information and associated text for annotation.
A method of querying a time series database, comprising:
- transforming a query to an embedded vector in a multi-modal shared latent space that encodes time series information and textual information;
- identifying a feature vector in the multi-modal shared latent space, stored in a time series dataspace, that matches the embedded vector, and that is associated with a data type complementary to the query;
- and
- returning data associated with the identified feature vector, responsive to the query.
A system for training a neural network, comprising:
- a hardware processor;
- and
- a memory that stores a computer program product, which, when executed by the hardware processor, causes the hardware processor to:
- train a time series embedding model and a text embedding model using unsupervised clustering to translate time series and text, respectively, to a multi-modal shared latent space;
- train the time series embedding model and the text embedding model further using semi-supervised clustering that samples training data pairs of time series information and associated text for annotation;
- transform a query to an embedded vector in the multi-modal shared latent space that encodes time series information and textual information;
- identifying a feature vector in the multi-modal shared latent space, stored in a time series dataspace, that matches the embedded vector, and that is associated with a data type complementary to the query;
- and
- returning data associated with the identified feature vector, responsive to the query.
Dependencies are inferred from the claim text.