US 12,762,090 B2Granted

Local large language model embedding search

AssigneeSap SE

InventorWenbo Wei

Life of this patent

Estimating the term…

Loading patent…

Abstract

In an example embodiment, a solution is provided that combines the precision of a database approach with the commonsense/smart approach of an LLM. Documents, such as Jira™ items, are each bound with a unique identification upon ingestion. The unique identification is used as a high-dimensional index to facilitate efficient search operations. Additionally, a local LLM model is used to process and analyze the data, which ensures that all data processing is kept locally. This helps prevent data exposure of confidential data contained in the files to external systems or networks. Finally, a secondary embedding search mechanism is implemented before presenting results to the user. The query is run more than once, and the outputs can then be compared. The results are only displayed if the match rate among the sets exceeds a predefined threshold. This enhances the precision of the LLM results, minimizing the risk of LLM-generated illusions or inaccuracies.

Claims20

Dependent claims

A system comprising:

at least one hardware processor;
and
a computer-readable medium storing instructions that, when executed by the at least one hardware processor, cause the at least one hardware processor to perform operations comprising:
accessing a project file stored in a file system, the project file describing a plurality of different requirements of a software project, the software project comprising a software development effort having defined functionality to be implemented, each requirement specifying a capability, feature, or constraint of the software project;
generating a separate requirements file for each requirement in the plurality of different requirements, each requirements file having a unique identification;
passing each requirements file through an embedding machine learning model to generate a corresponding requirements embedding stored in a vector database, the corresponding requirements embedding comprising a set of coordinates in a latent n-dimensional space such that proximity of coordinates reflects similarity of corresponding requirements files;
receiving a user query;
passing the user query through the embedding machine learning model to generate a query embedding;
searching the vector database to identify one or more requirements embeddings similar to the query embedding
generating a prompt based on the user query and based on the identified one or more requirements embeddings;
sending the prompt to a large language model (LLM) to cause the LLM to generate a natural language response to the user query based on the embeddings in the vector database;
receiving a first natural language response from the LLM;
repeating the sending;
receiving a second natural language response from the LLM;
validating the first natural language response based on a determination that the first natural language response matches the second natural language response by more than a threshold amount;
and
in response to the validating, causing the first natural language response to be displayed to a user.

14 more elements

A method comprising:

accessing a project file stored in a file system, the project file describing a plurality of different requirements of a software project, the software project comprising a software development effort having defined functionality to be implemented, each requirement specifying a capability, feature, or constraint of the software project;
generating a separate requirements file for each requirement in the plurality of different requirements, each requirements file having a unique identification;
passing each requirements file through an embedding machine learning model to generate a corresponding requirements embedding stored in a vector database, the corresponding requirements embedding comprising a set of coordinates in a latent n-dimensional space such that proximity of coordinates reflects similarity of corresponding requirements files;
receiving a user query;
passing the user query through the embedding machine learning model to generate a query embedding;
searching the vector database to identify one or more requirements embeddings similar to the query embedding generating a prompt based on the user query and based on the identified one or more requirements embeddings;
sending the prompt to a large language model (LLM) to cause the LLM to generate a natural language response to the user query based on the embeddings in the vector database;
receiving a first natural language response from the LLM;
repeating the sending;
receiving a second natural language response from the LLM;
validating the first natural language response based on a determination that the first natural language response matches the second natural language response by more than a threshold amount;
and
in response to the validating, causing the first natural language response to be displayed to a user.

10 more elements

A non-transitory machine-readable medium storing instructions which, when executed by one or more processors, cause the one or more processors to perform operations comprising:

accessing a project file stored in a file system, the project file describing a plurality of different requirements of a software project, the software project comprising a software development effort having defined functionality to be implemented, each requirement specifying a capability, feature, or constraint of the software project;
generating a separate requirements file for each requirement in the plurality of different requirements, each requirements file having a unique identification;
passing each requirements file through an embedding machine learning model to generate a corresponding requirements embedding stored in a vector database, the corresponding requirements embedding comprising a set of coordinates in a latent n-dimensional space such that proximity of coordinates reflects similarity of corresponding requirements files;
receiving a user query;
passing the user query through the embedding machine learning model to generate a query embedding;
searching the vector database to identify one or more requirements embeddings similar to the query embedding generating a prompt based on the user query and based on the identified one or more requirements embeddings;
sending the prompt to a large language model (LLM) to cause the LLM to generate a natural language response to the user query based on the embeddings in the vector database;
receiving a first natural language response from the LLM;
repeating the sending;
receiving a second natural language response from the LLM;
validating the first natural language response based on a determination that the first natural language response matches the second natural language response by more than a threshold amount;
and
in response to the validating, causing the first natural language response to be displayed to a user.

10 more elements

Dependencies are inferred from the claim text.