Vector lifecycle management in distributed storage systems

A computer-implemented method comprising:

• under control of a computing system comprising a computer-readable memory and a processor configured to execute specific instructions:
• receiving a vector to be inserted into a set of vectors, wherein the set of vectors is stored as a plurality of leaf objects of a distributed proximity-based graph across a plurality of storage devices of a distributed object storage system;
• identifying a subset of the plurality of leaf objects based at least partly on a similarity of the vector to one or more vectors stored in the subset of the plurality of leaf objects;
• storing the vector in an insert buffer associated with the subset of the plurality of leaf objects;
• and
• in response to a query;
• searching the insert buffer using a query vector at least partially in parallel with searching the distributed proximity-based graph using the query vector;
• and
• generating query results comprising the vector stored in the insert buffer, and at least one of the one or more vectors stored in the subset of the plurality of leaf objects.

A system comprising:

• computer-readable memory storing executable instructions;
• and
• one or more computing devices programmed by executable instructions to at least:
• receive a vector to be inserted into a set of vectors, wherein the set of vectors is stored as a plurality of leaf objects of a distributed proximity-based graph across a plurality of storage devices of a distributed object storage system;
• identify a subset of the plurality of leaf objects based at least partly on a similarity of the vector to one or more vectors stored in the subset of the plurality of leaf objects;
• store the vector in an insert buffer associated with the subset of the plurality of leaf objects;
• and
• in response to a query:
• search the insert buffer using a query vector at least partially in parallel with searching the distributed proximity-based graph using the query vector;
• and
• generate query results comprising the vector stored in the insert buffer, and at least one of the one or more vectors stored in the subset of the plurality of leaf objects.