Cluster aware caching for vector database
AssigneeAmazon Technologies, Inc.
InventorsMarc Brooker, Mrithyunjaya Kumar Annapragada
Life of this patent
Estimating the term…
Abstract
A vector-labeled database may improve a cache hit rate for read requests by performing a partial approximate nearest neighbor index search at a router to designate the cache that results of a read request are to be written to. The selected cache may be associated with a particular node of a layer of an approximate nearest neighbor index search. The router may direct all reads with search vectors close to the particular node to the associated cache.
Claims20
A system, comprising:
- a router;
- and
- one or more computing resources configured to implement a set of caches and a vector-labeled database, wherein results of a prior read request are added to a given cache in response to the prior read request;
- wherein the router is configured to:
- receive a read request for one or more data items associated with one or more approximate nearest neighbor vectors of a request vector, wherein the one or more approximate nearest neighbor vectors are maintained in the vector-labeled database;
- determine a portion of vector space for the request vector using at least a first layer of an approximate nearest neighbor index search;
- select, based on the determined portion of vector space for the request vector, a particular cache of the set of caches;
- cause the particular cache to be checked for the one or more approximate nearest neighbor vectors of the request vector;
- and
- cause the one or more data items associated with the one or more approximate nearest neighbor vectors to be returned in response to the request.
A method, comprising:
- receiving a read request for one or more data items associated with one or more approximate nearest neighbor vectors of a request vector;
- determining a portion of vector space for the request vector using at least a first layer of an approximate nearest neighbor index search;
- selecting, based on the determined portion of vector space for the request vector, a cache from a set of caches, wherein results of a prior read request are added to a given cache in response to the prior read request;
- and,
- causing the particular cache to be checked for the one or more approximate nearest neighbor vectors of the request vector.
A non-transitory computer-readable storage media storing program instructions, wherein the program instructions, when executed on or across one or more processors, cause the one or more processors to:
- receive a read request for one or more data items associated with one or more approximate nearest neighbor vectors of a request vector;
- determine a portion of vector space for the request vector using at least a first layer of an approximate nearest neighbor index search;
- and
- select, based on the determined portion of vector space for the request vector, a cache from a set of caches, wherein results of a prior read request are added to a given cache in response to the prior read request;
- and
- cause the particular cache to be checked for the one or more approximate nearest neighbor vectors of the request vector.
Dependencies are inferred from the claim text.