US 12,743,578 B2Granted

Large language models for microservice fault recovery

AssigneeSap SE

InventorHui Li

Life of this patent

Estimating the term…

Loading patent…

Abstract

In an example embodiment, an LLM is used to identify similar microservices to allow a microservice that is similar to a dependent microservice that is down to be used instead of the downed dependent microservice until the downed dependent microservice can be brought back online. Specifically, the LLM is utilized in two different manners. First, it is used to generate an embedding for an API of each of multiple microservices in a system. These embeddings may then be used to retrieve similar APIs to the API of a downed microservice. Then the LLM can be further used to select the most qualified of the similar APIs, based on similar functionality and input/output parameters. The most qualified of the APIs can then be tested for final selection of a (temporary) replacement API that can be used in lieu of the API of the downed microservice.

Claims20

Dependent claims

A system comprising:

at least one hardware processor;
a non-tangible computer-readable medium storing instructions that, when executed by the at least one hardware processor, cause the at least one hardware processor to perform operations comprising:
generating a first prompt comprising a request to generate an embedding for each of a plurality of document fragments of application program interfaces (APIs) for microservices, the embedding comprising a coordinate in a latent n-dimensional space;
sending the first prompt to a large language model (LLM);
receiving a plurality of embeddings from the LLM;
for a first document fragment, calculating cosine correlation coefficients between the first document fragment and each of a plurality of other document fragments, using the plurality of embeddings;
based on the cosine correlation coefficients, selecting a set of candidate document fragments;
generating a second prompt comprising an identification of the set of candidate document fragments and a request to identify qualified APIs corresponding to one or more candidate fragments in the set of candidate document fragments;
sending the second prompt to the large language model (LLM);
receiving an indication of a set of qualified APIs;
determining a first downstream microservice with a first API is down;
and
in response to the determining, causing a first upstream microservice to use one API in the set of qualified APIs in conjunction with using a second downstream microservice.

10 more elements

A non-transitory machine-readable medium storing instructions which, when executed by one or more processors, cause the one or more processors to perform operations comprising:

generating a first prompt comprising a request to generate an embedding for each of a plurality of document fragments of application program interfaces (APIs) for microservices, the embedding comprising a coordinate in a latent n-dimensional space;
sending the first prompt to a large language model (LLM);
receiving a plurality of embeddings from the LLM;
for a first document fragment, calculating cosine correlation coefficients between the first document fragment and each of a plurality of other document fragments, using the plurality of embeddings;
based on the cosine correlation coefficients, selecting a set of candidate document fragments;
generating a second prompt comprising an identification of the set of candidate document fragments and a request to identify qualified APIs corresponding to one or more candidate fragments in the set of candidate document fragments;
sending the second prompt to the large language model (LLM);
receiving an indication of a set of qualified APIs;
determining a first downstream microservice with a first API is down;
and
in response to the determining, causing a first upstream microservice to use one API in the set of qualified APIs in conjunction with using a second downstream microservice.

8 more elements

Dependencies are inferred from the claim text.