Semantic segmentation of data for information retrieval

A method for retrieving information from an auxiliary database that includes a vector database and a knowledge graph, the method comprising:

• receiving a query from a client, wherein the query includes a semantic type;
• generating a query embedding from the query;
• searching the vector database with the query embedding to identify a k most similar chunk embeddings to the query embedding;
• identifying chunks corresponding to the chunk embeddings and their corresponding nodes in the knowledge graph;
• obtaining semantic cells associated to the identified chunks and an object associated with the semantic cells;
• obtaining metadata cells associated with the object and identifying a set of the metadata cells whose semantic type matches the semantic type included in the query;
• and
• returning a return object that includes the object, the semantic cells, and the set of metadata cells.

A non-transitory storage medium having stored therein instructions that are executable by one or more hardware processors to perform operations comprising:

• accessing an object stored in a repository of objects;
• dividing the object into distinct parts;
• labelling each of the distinct parts as a semantic cell, a metadata cell, or irrelevant such that the distinct parts include semantic cells and metadata cells;
• splitting content of each of the semantic cells into chunks;
• generating chunk embeddings for each of the chunks of each of the semantic cells;
• and
• generating a knowledge graph that includes a root node representing the repository,
• wherein:
• the object is represented by an object node under the root node,
• the semantic cells and metadata cells are arranged as semantic nodes and metadata nodes hierarchically under the object node,
• chunk nodes are created for each of the chunks under corresponding semantic nodes, and
• each of the chunk embeddings includes a pointer to a corresponding chunk node in the knowledge graph.