Systems and methods for trained embedding mappings for improved retrieval augmented generation

A method, comprising:

• storing, by one or more processors coupled to non-transitory memory, a dataset comprising a first set of embeddings corresponding to a first embeddings space and stored in association with a set of query results for the first set of embeddings, the set of query results corresponding to a second embeddings space;
• training, by the one or more processors, a transformation data structure using the first set of embeddings and the set of query results, the transformation data structure to transform the first set of embeddings to the second embeddings space, wherein the first embeddings space is different from the second embeddings space;
• and
• executing, by the one or more processors, a search operation for the second embeddings space by applying the transformation data structure to a second set of embeddings corresponding to the first embeddings space.

A system, comprising:

• one or more processors coupled to non-transitory memory, the one or more processors configured to:
• store a dataset comprising a first set of embeddings corresponding to a first embeddings space and stored in association with a set of query results for the first set of embeddings, the set of query results corresponding to a second embeddings space;
• train a transformation data structure using the first set of embeddings and the set of query results, the transformation data structure to transform the first set of embeddings to the second embeddings space, wherein the second embeddings space is a subset of the first embeddings space;
• and
• execute a search operation for the second embeddings space by applying the transformation data structure to a second set of embeddings corresponding to the first embeddings space.

A method, comprising:

• storing, by one or more processors coupled to non-transitory memory, a dataset comprising a first set of embeddings corresponding to a first embeddings space and stored in association with a set of query results for the first set of embeddings, the set of query results corresponding to a second embeddings space;
• training, by the one or more processors, a transformation data structure using the first set of embeddings and the set of query results, the transformation data structure to transform the first set of embeddings to the second embeddings space, wherein the second embeddings space is a subset of the first embeddings space;
• and
• executing, by the one or more processors, a search operation for the second embeddings space by applying the transformation data structure to a second set of embeddings corresponding to the first embeddings space.