Systems and methods for providing self-improving artificial intelligence models

A system for generating reasoning structures for a task performed by an artificial intelligence (AI) agent, the system comprising:

• a memory that stores a neural network-based language model associated with the AI agent and a plurality of processor executable instructions;
• a communication interface that receives training data samples;
• and
• one or more hardware processors that read and execute the plurality of processor-executable instructions from the memory to perform operations comprising:
• generating, by the neural network-based language model of the AI agent, a plurality of reasoning paths relating to a task request based on a plurality of task-agnostic reasoning guidelines, wherein each reasoning path in the plurality of reasoning paths includes a sequence of steps that indicate a logic flow in solving the task request;
• generating, by the neural network-based language model, a plurality of solutions for the task request according to the plurality of reasoning paths, wherein each of the plurality of solutions comprises an answer to the task request based on an application of a corresponding reasoning path from the plurality of reasoning paths to the task request;
• generating a plurality of training datasets for training the neural network-based language model based on the plurality of reasoning paths and the plurality of solutions, wherein each training dataset in the plurality of training datasets comprises at least the task request, a corresponding reasoning path from the plurality of reasoning paths, and a benchmark answer;
• and
• training the neural network-based language model based on the plurality of training dataset.

A non-transitory machine-readable medium comprising a plurality of machine-executable instructions which, when executed by one or more processors, are adapted to cause the one or more processors to perform operations comprising:

• generating, by a neural network-based language model of the AI agent, a plurality of reasoning paths relating to a task request based on a plurality of task-agnostic reasoning guidelines, wherein each reasoning path in the plurality of reasoning paths includes a sequence of steps that indicate a logic flow in solving the task request;
• generating, by the neural network-based language model, a plurality of solutions for the task request according to the plurality of reasoning paths, wherein each of the plurality of solutions comprises an answer to the task request based on an application of a corresponding reasoning path from the plurality of reasoning paths to the task request;
• generating a plurality of training datasets for training the neural network-based language model based on the plurality of reasoning paths and the plurality of solutions, wherein each training dataset in the plurality of training datasets comprises at least the task request, a corresponding reasoning path from the plurality of reasoning paths, and a benchmark answer;
• and
• training the neural network-based language model based on the plurality of training dataset.