Selecting indexing algorithms for automated embedding database generation

A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform operations comprising:

• accessing, by an online system, a set of embedding entries and a set of parameters for an embedding database, wherein embedding entries comprise an embedding and an identifier for an object associated with the embedding, and wherein the set of parameters comprise a performance parameter for a content-serving subsystem of the online system, wherein the content-serving subsystem is a system that uses embeddings from the set of embedding entries, and wherein the performance parameter represents an performance of the content-serving subsystem while using embedding from the set of embedding entries;
• accessing, by the online system, a set of candidate indexing algorithms for the embedding database, wherein each candidate indexing algorithm is an indexing algorithm for indexing embeddings within a database;
• generating, by the online system, an algorithm score for each candidate indexing algorithm, wherein an algorithm score represents a measure of compliance with the set of parameters of the corresponding candidate indexing algorithm, wherein generating an algorithm score for a candidate indexing algorithm comprises:
• initializing, by the online system, a testing embedding database in accordance with the candidate indexing algorithm;
• and
• performing, by the online system, a set of benchmarking tests on the testing embedding database to generate one or more performance metrics for the candidate indexing algorithm, wherein performing the set of benchmark tests comprises:
• accessing a plurality of testing examples, wherein each testing example of the plurality of testing examples comprises input data for the content-serving subsystem and a label indicating an expected output of the content-serving subsystem for the corresponding input data;
• for each of the plurality of testing examples:
• inputting the input data to the content-serving subsystem;
• executing the content-serving subsystem using the testing embedding database;
• receiving an output from the content-serving subsystem;
• and
• comparing the output to the label of the testing example;
• and
• generating the one or more performance metrics based on the comparisons of the outputs to the labels of the plurality of testing examples;
• selecting, by the online system, an indexing algorithm of the set of candidate indexing algorithm based on the generated algorithm scores;
• and
• storing, by the online system, the set of embedding entries in the embedding database using the selected indexing algorithm.

A system comprising:

• a processor;
• and
• a non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform operations comprising:
• accessing, by an online system, a set of embedding entries and a set of parameters for an embedding database, wherein embedding entries comprise an embedding and an identifier for an object associated with the embedding, and wherein the set of parameters comprise a performance parameter for a content-serving subsystem of the online system, wherein the content-serving subsystem is a system that uses embeddings from the set of embedding entries, and wherein the performance parameter represents an performance of the content-serving subsystem while using embedding from the set of embedding entries;
• accessing, by the online system, a set of candidate indexing algorithms for the embedding database, wherein each candidate indexing algorithm is an indexing algorithm for indexing embeddings within a database;
• generating, by the online system, an algorithm score for each candidate indexing algorithm, wherein an algorithm score represents a measure of compliance with the set of parameters of the corresponding candidate indexing algorithm, wherein generating an algorithm score for a candidate indexing algorithm comprises:
• initializing, by the online system, a testing embedding database in accordance with the candidate indexing algorithm;
• and
• performing, by the online system, a set of benchmarking tests on the testing embedding database to generate one or more performance metrics for the candidate indexing algorithm, wherein performing the set of benchmark tests comprises:
• accessing a plurality of testing examples, wherein each testing example of the plurality of testing examples comprises input data for the content-serving subsystem and a label indicating an expected output of the content-serving subsystem for the corresponding input data;
• for each of the plurality of testing examples:
• inputting the input data to the content-serving subsystem;
• executing the content-serving subsystem using the testing embedding database;
• receiving an output from the content-serving subsystem;
• and
• comparing the output to the label of the testing example;
• and
• generating the one or more performance metrics based on the comparisons of the outputs to the labels of the plurality of testing examples;
• selecting, by the online system, an indexing algorithm of the set of candidate indexing algorithm based on the generated algorithm scores;
• and
• storing, by the online system, the set of embedding entries in the embedding database using the selected indexing algorithm.