Efficient embedding table storage and lookup

#The present disclosure generally relates to machine learning. More particularly, the present disclosure relates to providing efficient embedding table storage and lookup in machine-learning models.

#Embeddings generally refer to a way of projecting objects from a high-dimensional space onto a lower-dimensional space. For example, image, video, text, and speech data may be represented in an object space, such as a two-dimensional or three-dimensional coordinate system. Each data point in the object space may represent an object with the proximity between each object indicating a degree of similarity or relatedness. In one example, synonyms generally may be positioned more closely together in the object space in comparison to unassociated or dissimilar words.

#Machine learning models may utilize embedding tables to store and reference embeddings. Embedding tables become part of a machine learning model and embedding tables may be quite large in size. For example, embedding tables for dictionary words from one or more languages may have hundreds of thousands or even millions of entries. In particular, embedding tables can significantly increase the amount of memory and processing power required to run machine-learning models. As a result, it becomes increasingly difficult to serve large models at scale. Further, many types of computing devices, including mobile devices, generally lack enough computing resources to run large models.

#Detailed discussion of embodiments directed to one of ordinary skill in the art is set forth in the specification, which makes reference to the appended figures, in which:

#FIG. 1A depicts a block diagram of an example system for providing efficient embedding table storage and lookup in machine-learning models according to example embodiments of the present disclosure.

#FIG. 1B depicts a block diagram of an example embedding table for providing efficient embedding table storage and lookup in machine-learning models according to example embodiments of the present disclosure.

#Reference numerals that are repeated across plural figures are intended to identify the same features in various implementations.

#Generally, the present disclosure is directed to providing efficient embedding table storage and lookup for machine-learning models. Examples described in the present disclosure enable the efficient storage and retrieval of embedding tables used with machine-learning models. Such example implementations provide advantages and improvements over existing approaches including, but not limited to, improved scalability, increased performance, and reduced computational requirements as compared to existing approaches.

#Embedding tables are used in machine learning as a component of machine learning models. For example, embedding tables may be used in machine-learning models related to natural language processing (NLP) or other various implementations. Embedding tables assist in constructing abstract relationships between objects (e.g., words, phrases, images, songs, movies, etc.) and include embeddings that project such objects from a high-dimensional space onto a lower-dimensional space. For example, words, phrases, images, songs, movies, or any other type of object may be represented in an object space, such as a two-dimensional or three-dimensional coordinate system, where proximity between the objects represents a level of relatedness between the objects. In various examples, an object space may be any number of dimensions and are not limited solely to two or three dimensions. For example, an object space may have more than three dimensions, more than ten dimensions, more than one hundred dimensions or generally any number of dimensions. In addition, embedding tables can be trained, for example, within a machine-learning model, and embedding tables may be reused across different machine-learning models.