US 12,608,657 B1Granted

Automated continual learning system for machine learning models

AssigneeIntuit Inc.

InventorsXiang Gao, Yuguang Yao, Kamalika Das, Avinash Baidya, Ruocheng Guo

Life of this patent

Estimating the term…

Loading patent…

Abstract

Techniques described herein involve providing an automated continual learning system for machine learning models. Embodiments include generating, using a machine learning model, an output based on a sample input provided to the machine learning model. Embodiments include evaluating the output based on comparing the output to an associated training output. Embodiments include creating a natural language rule based on the evaluating using a language processing machine learning model and storing the natural language rule in a rule library. Embodiments include receiving an input to the machine learning model and retrieving, in response to the input, one or more relevant rules from the rule library. Embodiments include generating, using the machine learning model, a response based on the input and the one or more relevant rules and performing an action based on the response.

Claims11

Dependent claims

A method for providing an automated continual learning system for machine learning models, comprising:

generating, using a machine learning model, an output based on a sample input provided to the machine learning model;
evaluating the output based on comparing the output to an associated training output, identifying discrepancies between the output and the associated training output, and generating a natural language explanation for the discrepancies;
creating a natural language rule based on the evaluating using a language processing machine learning model, wherein the creating comprises reformulating the natural language explanation into the natural language rule, and wherein the reformulating comprises altering content and linguistic characteristics of the natural language explanation using the language processing machine learning model;
storing the natural language rule in a rule library;
receiving an input to the machine learning model;
retrieving, in response to the input, one or more relevant rules from the rule library based on determining semantic similarity between the input and a subset of available rules stored in the rule library;
generating, using the machine learning model, a response based on the input and the one or more relevant rules, wherein the one or more relevant rules, as a result of being retrieved based on the determining of the semantic similarity, enable the machine learning model to generate the response with at least a same level of accuracy without fine-tuning the machine learning model;
and
performing an action based on the response.

6 more elements

A system for providing an automated continual learning system for machine learning models, comprising:

one or more processors;
and
a memory comprising instructions that, when executed by the one or more processors, cause the system to:
generate, using a machine learning model, an output based on a sample input provided to the machine learning model;
evaluate the output based on comparing the output to an associated training output, identifying discrepancies between the output and the associated training output, and generating a natural language explanation for the discrepancies;
create a natural language rule based on the evaluating using a language processing machine learning model, wherein the creating comprises reformulating the natural language explanation into the natural language rule, and wherein the reformulating comprises altering content and linguistic characteristics of the natural language explanation using the language processing machine learning model;
store the natural language rule in a rule library;
receive an input to the machine learning model;
retrieve, in response to the input, one or more relevant rules from the rule library based on determining semantic similarity between the input and a subset of available rules stored in the rule library;
generate, using the machine learning model, a response based on the input and the one or more relevant rules, wherein the one or more relevant rules, as a result of being retrieved based on the determining of the semantic similarity, enable the machine learning model to generate the response with at least a same level of accuracy without fine-tuning the machine learning model;
and
perform an action based on the response.

9 more elements

A non-transitory computer readable medium comprising instructions that, when executed by one or more processors of a computing system, cause the computing system to:

generate, using a machine learning model, an output based on a sample input provided to the machine learning model;
evaluate the output based on comparing the output to an associated training output, identifying discrepancies between the output and the associated training output, and generating a natural language explanation for the discrepancies;
create a natural language rule based on the evaluating using a language processing machine learning model, wherein the creating comprises reformulating the natural language explanation into the natural language rule, and wherein the reformulating comprises altering content and linguistic characteristics of the natural language explanation using the language processing machine learning model;
store the natural language rule in a rule library;
receive an input to the machine learning model;
retrieve, in response to the input, one or more relevant rules from the rule library based on determining semantic similarity between the input and a subset of available rules stored in the rule library;
generate, using the machine learning model, a response based on the input and the one or more relevant rules, wherein the one or more relevant rules, as a result of being retrieved based on the determining of the semantic similarity, enable the machine learning model to generate the response with at least a same level of accuracy without fine-tuning the machine learning model;
and
perform an action based on the response.

6 more elements

Dependencies are inferred from the claim text.