Method, System, and Computer Program Product for Embedding Compression and Regularization

A system for generating a compressed representation of embedding vector data comprising at least one processor programmed or configured to:

• receive the embedding vector data comprising a plurality of embedding vectors;
• train a beta-variational autoencoder based on the embedding vector data and a loss equation, the beta-variational autoencoder comprising an encoder network, a latent layer, and a decoder network, the loss equation comprising a first term associated with reconstruction of an input by the beta-variational autoencoder, a second term associated with regularization of a latent space of the latent layer, and a hyperparameter weight associated with a ratio of the first term and the second term, wherein the latent space has a plurality of dimensions;
• determine a respective mean of each respective dimension of the plurality of dimensions;
• determine a respective variance of each respective dimension of the plurality of dimensions;
• determine a first subset of the plurality of dimensions based on the respective mean and the respective variance for each respective dimension of the plurality of dimensions;
• discard a second subset of the plurality of dimensions based on the respective mean and the respective variance of each respective dimension of the plurality of dimensions;
• and
• generate the compressed representation of the embedding vector data based on the first subset of dimensions.

A computer program product for generating a compressed representation of embedding vector data, the computer program product comprising at least one non-transitory computer readable medium including one or more instructions that, when executed by at least one processor, cause the at least one processor to:

• receive the embedding vector data comprising a plurality of embedding vectors;
• train a beta-variational autoencoder based on the embedding vector data and a loss equation, the beta-variational autoencoder comprising an encoder network, a latent layer, and a decoder network, the loss equation comprising a first term associated with reconstruction of an input by the beta-variational autoencoder, a second term associated with regularization of a latent space of the latent layer, and a hyperparameter weight associated with a ratio of the first term and the second term, wherein the latent space has a plurality of dimensions;
• determine a respective mean of each respective dimension of the plurality of dimensions;
• determine a respective variance of each respective dimension of the plurality of dimensions;
• determine a first subset of the plurality of dimensions based on the respective mean and the respective variance for each respective dimension of the plurality of dimensions;
• discard a second subset of the plurality of dimensions based on the respective mean and the respective variance of each respective dimension of the plurality of dimensions;
• and
• generate the compressed representation of the embedding vector data based on the first subset of dimensions.