Real-time adaptation of machine learning models using large language models

A non-transitory computer-readable storage medium coupled to one or more processors and having instructions stored thereon which, when executed by the one or more processors, cause the one or more processors to perform operations for real-time monitoring of performance of a machine learning (ML) model and selectively triggering one of fine-tuning and re-training of the ML model, the operations comprising:

• receiving a query entity and a target entity pair that are input to the ML model;
• receiving an inference result output from the ML model for the query entity and the target entity pair;
• generating a detection prompt at least partially based on unstructured data determined from one or more of the query entity and the target entity;
• determining an unstructured data pattern vector that is representative of an output of a large language model (LLM) that is responsive to the detection prompt;
• providing a data pattern metric based on the unstructured data pattern vector;
• generating an explanation prompt at least partially based on the inference result;
• determining a correctness result that is representative of a correctness of the inference result using explanation text that is generated by the LLM and is responsive to the explanation prompt;
• in response to determining that there is a threshold decrease in accuracy at least partially based on the correctness result, automatically executing one of fine-tuning of the ML model and re-training of the ML model responsive to the data pattern metric to provide an adapted ML model;
• and
• deploying the adapted ML model for inference.

A system, comprising:

• a computing device;
• and
• a computer-readable storage device coupled to the computing device and having instructions stored thereon which, when executed by the computing device, cause the computing device to perform operations for real-time monitoring of performance of a machine learning (ML) model and selectively triggering one of fine-tuning and re-training of the ML model, the operations comprising:
• receiving a query entity and a target entity pair that are input to the ML model;
• receiving an inference result output from the ML model for the query entity and the target entity pair;
• generating a detection prompt at least partially based on unstructured data determined from one or more of the query entity and the target entity;
• determining an unstructured data pattern vector that is representative of an output of a large language model (LLM) that is responsive to the detection prompt;
• providing a data pattern metric based on the unstructured data pattern vector;
• generating an explanation prompt at least partially based on the inference result;
• determining a correctness result that is representative of a correctness of the inference result using explanation text that is generated by the LLM and is responsive to the explanation prompt;
• in response to determining that there is a threshold decrease in accuracy at least partially based on the correctness result, automatically executing one of fine-tuning of the ML model and re-training of the ML model responsive to the data pattern metric to provide an adapted ML model;
• and
• deploying the adapted ML model for inference.