Systems and methods for a neural network language model
AssigneeSalesforce, Inc.
InventorsShafiq Rayhan Joty, Xuan Phi Nguyen, Yifei Ming, Zixuan Ke
Life of this patent
Estimating the term…
Abstract
Embodiments described herein provide a multi-stage training and/or post-training framework to train and/or finetune a GLLM for domain-specific tasks so as to build an AI agent in a variety of technical applications. Specifically, the training framework comprises a first stage of combined continual pretraining (CPT) and instruction tuning (IT), and a second state of preference training.
Claims20
A method of building an artificial intelligence (AI) agent using a neural network based language model, the method comprising:
- obtaining, via a data interface, a training dataset comprising a question, a solution to the question, and a reasoning path comprising multiple steps to result in the solution;
- generating, by a first neural network based language model, a first step from the multiple steps that is erroneous based on an input of the question and the reasoning path;
- generating, by the first neural network based language model, a corrected step in place of the first step based on an input prompt of the question and steps up to the first step;
- constructing a preference training dataset comprising the input prompt, the generated corrected step as a positive sample, and the first step as a negative sample;
- training a second neural network based language model using the preference training dataset through preference learning;
- and
- building the AI agent based on the trained neural network based language model to generate a task solution and a task reasoning path that result in the task solution to a user input request.
A system of building an artificial intelligence (AI) agent using a neural network based language model, the system comprising:
- a data interface obtaining a training dataset comprising a question, a solution to the question, and a reasoning path comprising multiple steps to result in the solution;
- a memory storing a plurality of processor-executable instructions;
- and
- one or more processors executing the plurality of processor-executable instructions to perform operations comprising:
- generating, by a first neural network based language model, a first step from the multiple steps that is erroneous based on an input of the question and the reasoning path;
- generating, by the first neural network based language model, a corrected step in place of the first step based on an input prompt of the question and steps up to the first step;
- constructing a preference training dataset comprising the input prompt, the generated corrected step as a positive sample, and the first step as a negative sample;
- training a second neural network based language model using the preference training dataset through preference learning;
- and
- building the AI agent based on the trained neural network based language model to generate a task solution and a task reasoning path that result in the task solution to a user input request.
A non-transitory processor-readable medium storing a plurality of processor-executable instructions for building an artificial intelligence (AI) agent using a neural network based language model, the instructions executable by one or more processors to perform operations comprising:
- obtaining, via a data interface, a training dataset comprising a question, a solution to the question, and a reasoning path comprising multiple steps to result in the solution;
- generating, by a first neural network based language model, a first step from the multiple steps that is erroneous based on an input of the question and the reasoning path;
- generating, by the first neural network based language model, a corrected step in place of the first step based on an input prompt of the question and steps up to the first step;
- constructing a preference training dataset comprising the input prompt, the generated corrected step as a positive sample, and the first step as a negative sample;
- training a second neural network based language model using the preference training dataset through preference learning;
- and
- building the AI agent based on the trained neural network based language model to generate a task solution and a task reasoning path that result in the task solution to a user input request.
Dependencies are inferred from the claim text.