# US 2026/0094003 A1 · Systems and methods for a neural network language model

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0094003 A1
- Document ID: US20260094003A1
- Status: Published application
- Kind: A1
- Application number: 19/043,184
- Filed: 2025-01-31
- Published: 2026-04-02
- Earliest claimed: 2024-09-30
- Assignees: Salesforce, Inc.
- Applicants: Salesforce, Inc.
- Inventors: Shafiq Rayhan Joty; Xuan Phi Nguyen; Yifei Ming; Zixuan Ke
- Web page: https://patentgrep.com/patent/US20260094003A1

## CPC

- G06N 3/096: Transfer learning
- G06F 40/284: Lexical analysis, e.g. tokenisation or collocates
- G06N 3/045: Combinations of networks
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Embodiments described herein provide a multi-stage training and/or post-training framework to train and/or finetune a GLLM for domain-specific tasks so as to build an AI agent in a variety of technical applications. Specifically, the training framework comprises a first stage of combined continual pretraining (CPT) and instruction tuning (IT), and a second state of preference training.

## Claims

### Claim 1

A method of building an artificial intelligence (AI) agent using a neural network based language model, the method comprising:
obtaining, via a data interface, a training dataset comprising a question, a solution to the question, and a reasoning path comprising multiple steps to result in the solution;
generating, by a first neural network based language model, a first step from the multiple steps that is erroneous based on an input of the question and the reasoning path;
generating, by the first neural network based language model, a corrected step in place of the first step based on an input prompt of the question and steps up to the first step;
constructing a preference training dataset comprising the input prompt, the generated corrected step as a positive sample, and the first step as a negative sample;
training a second neural network based language model using the preference training dataset through preference learning; and
building the AI agent based on the trained neural network based language model to generate a task solution and a task reasoning path that result in the task solution to a user input request.

### Claim 2 (depends on 1)

The method of claim 1, wherein the reasoning path is generated by the first neural network based language model based on an input of the question and the solution.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
generating, by the first neural network based language model, a binary decision indicating whether the solution is accurate; and
constructing another preference training dataset comprising the question, a positive sample of the solution that is determined to be accurate, or a negative sample of the solution that is determined to be inaccurate.

### Claim 4 (depends on 1)

The method of claim 1, wherein the second neural network based language model is trained based on a preference loss that maximizes a difference between a first model-generated logit corresponding to the positive sample, and a second model-generated logit corresponding to the negative sample.

### Claim 5 (depends on 1)

The method o claim 1, wherein the second neural network based language model is pretrained, prior to the preference learning by a mixed dataset of task-specific, non-task specific, domain-specific and non-domain specific texts.

### Claim 6 (depends on 5)

The method of claim 5, wherein the pretraining comprises:
constructing a first training dataset comprising one or more text samples;
constructing a second training dataset comprising one or more instructional samples, at least one instructional sample comprising a question, an answer and an instruction instructing the neural network based language model to perform a specific task resulting in the answer;
constructing a third training dataset by mixing the first training dataset and a second training dataset using a pre-defined mixture ratio;
randomly selecting a training sample from the third training dataset;
generating, by the first neural network based language model, a predicted answer to the question conditioned on the instruction when the randomly selected training sample belongs to the second training dataset;
generating, by the first neural network based language model, a reconstructed text in response to a text sample when the randomly selected training sample belongs to the first training dataset; and
jointly training the second neural network based language model based on a first loss comparing the predicted answer to the answer, and a second loss comparing the reconstructed text and the text sample over one or more training iterations.

### Claim 7 (depends on 6)

The method of claim 6, wherein the second neural network based language model is alternately updated based on the first loss or the second loss over the one or more training iterations.

### Claim 8 (depends on 6)

The method of claim 6, wherein the user input request comprises an unseen task prompt not included in the first training dataset or the second training dataset.

### Claim 9

A system of building an artificial intelligence (AI) agent using a neural network based language model, the system comprising:
a data interface obtaining a training dataset comprising a question, a solution to the question, and a reasoning path comprising multiple steps to result in the solution;
a memory storing a plurality of processor-executable instructions; and
one or more processors executing the plurality of processor-executable instructions to perform operations comprising:
generating, by a first neural network based language model, a first step from the multiple steps that is erroneous based on an input of the question and the reasoning path;
generating, by the first neural network based language model, a corrected step in place of the first step based on an input prompt of the question and steps up to the first step;
constructing a preference training dataset comprising the input prompt, the generated corrected step as a positive sample, and the first step as a negative sample;
training a second neural network based language model using the preference training dataset through preference learning; and
building the AI agent based on the trained neural network based language model to generate a task solution and a task reasoning path that result in the task solution to a user input request.

### Claim 10 (depends on 9)

The system of claim 9, wherein the reasoning path is generated by the first neural network based language model based on an input of the question and the solution.

### Claim 11 (depends on 9)

The system of claim 9, wherein the operations further comprise:
generating, by the first neural network based language model, a binary decision indicating whether the solution is accurate; and
constructing another preference training dataset comprising the question, a positive sample of the solution that is determined to be accurate, or a negative sample of the solution that is determined to be inaccurate.

### Claim 12 (depends on 9)

The system of claim 9, wherein the second neural network based language model is trained based on a preference loss that maximizes a difference between a first model-generated logit corresponding to the positive sample, and a second model-generated logit corresponding to the negative sample.

### Claim 13 (depends on 9)

The system o claim 9, wherein the second neural network based language model is pretrained, prior to the preference learning by a mixed dataset of task-specific, non-task specific, domain-specific and non-domain specific texts.

### Claim 14 (depends on 13)

The system of claim 13, wherein the operation of pretraining comprises:
constructing a first training dataset comprising one or more text samples;
constructing a second training dataset comprising one or more instructional samples, at least one instructional sample comprising a question, an answer and an instruction instructing the neural network based language model to perform a specific task resulting in the answer;
constructing a third training dataset by mixing the first training dataset and a second training dataset using a pre-defined mixture ratio;
randomly selecting a training sample from the third training dataset;
generating, by the first neural network based language model, a predicted answer to the question conditioned on the instruction when the randomly selected training sample belongs to the second training dataset;
generating, by the first neural network based language model, a reconstructed text in response to a text sample when the randomly selected training sample belongs to the first training dataset; and
jointly training the second neural network based language model based on a first loss comparing the predicted answer to the answer, and a second loss comparing the reconstructed text and the text sample over one or more training iterations.

### Claim 15 (depends on 14)

The system of claim 14, wherein the second neural network based language model is alternately updated based on the first loss or the second loss over the one or more training iterations.

### Claim 16 (depends on 14)

The system of claim 14, wherein the user input request comprises an unseen task prompt not included in the first training dataset or the second training dataset.

### Claim 17

A non-transitory processor-readable medium storing a plurality of processor-executable instructions for building an artificial intelligence (AI) agent using a neural network based language model, the instructions executable by one or more processors to perform operations comprising:
obtaining, via a data interface, a training dataset comprising a question, a solution to the question, and a reasoning path comprising multiple steps to result in the solution;
generating, by a first neural network based language model, a first step from the multiple steps that is erroneous based on an input of the question and the reasoning path;
generating, by the first neural network based language model, a corrected step in place of the first step based on an input prompt of the question and steps up to the first step;
constructing a preference training dataset comprising the input prompt, the generated corrected step as a positive sample, and the first step as a negative sample;
training a second neural network based language model using the preference training dataset through preference learning; and
building the AI agent based on the trained neural network based language model to generate a task solution and a task reasoning path that result in the task solution to a user input request.

### Claim 18 (depends on 17)

The non-transitory processor-readable medium of claim 17, wherein the reasoning path is generated by the first neural network based language model based on an input of the question and the solution.

### Claim 19 (depends on 17)

The non-transitory processor-readable medium of claim 17, wherein the operations further comprise:
generating, by the first neural network based language model, a binary decision indicating whether the solution is accurate; and
constructing another preference training dataset comprising the question, a positive sample of the solution that is determined to be accurate, or a negative sample of the solution that is determined to be inaccurate.

### Claim 20 (depends on 17)

The non-transitory processor-readable medium of claim 17, wherein the second neural network based language model is trained based on a preference loss that maximizes a difference between a first model-generated logit corresponding to the positive sample, and a second model-generated logit corresponding to the negative sample.
