Systems and methods for training a language processing model

A method for training a language processing model for a chatbot, comprising:

• receiving a first rule set to regulate training a language processing model, wherein the first rule set includes one or more activation patterns, and wherein the one or more activation patterns are associated with translation of input text sequences into output text sequences by the language processing model;
• training the language processing model using a training dataset associated with the chatbot and the first rule set to produce an output text sequence, wherein the language processing model is trained with a first loss function that rewards the language processing model for adherence to the first rule set;
• generating a first performance metric for the language processing model as a result of the training, wherein the first performance metric is based on the first loss function and a second loss function that rewards the language processing model for producing output text sequences in accordance with the training dataset;
• using a genetic algorithm, generating a second rule set based on the first rule set, wherein the genetic algorithm uses symbolic syntax to evolve an input rule set into an output rule set;
• using a reinforcement learning algorithm and the second rule set, updating parameters of the language processing model to generate an updated language processing model, wherein the reinforcement learning algorithm evaluates parameters of the language processing model for adherence to the second rule set;
• generating a second performance metric for the updated language processing model, wherein the second performance metric is based on the first loss function and the second loss function;
• based on the first performance metric and the second performance metric, using the reinforcement learning algorithm to generate an updated genetic algorithm, wherein the reinforcement learning algorithm evaluates a plurality of configurations of symbolic syntax for the genetic algorithm to optimize the first performance metric and the second performance metric;
• using the updated genetic algorithm, generating a third rule set based on the second rule set;
• using the reinforcement learning algorithm and the third rule set to generate a final language processing model based on the updated language processing model;
• and
• using the final language processing model to generate a set of text responses to a set of queries.

One or more non-transitory, computer-readable media comprising instructions that, when executed by one or more processors, cause operations comprising:

• receiving a language processing model and a first rule set to regulate the language processing model;
• generating a first performance metric for the language processing model based on a first loss function that rewards the language processing model for adherence to the first rule set;
• using a genetic algorithm, generating a second rule set based on the first rule set;
• using a reinforcement learning algorithm and the second rule set, updating parameters of the language processing model to generate an updated language processing model, wherein the reinforcement learning algorithm evaluates parameters of the language processing model for adherence to the second rule set;
• generating a second performance metric for the updated language processing model, wherein the second performance metric is based on the first loss function;
• based on the first performance metric and the second performance metric, using the reinforcement learning algorithm to generate an updated genetic algorithm;
• using the updated genetic algorithm, generating a third rule set based on the second rule set;
• using a reinforcement learning algorithm and the third rule set to generate a final language processing model based on the updated language processing model;
• and
• using the final language processing model to generate a set of text responses to a set of queries.