# US 2025/0094700 A1 · Small sample fine-turning method and system and related apparatus

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0094700 A1
- Document ID: US20250094700A1
- Status: Published application
- Kind: A1
- Application number: 18/724,632
- Filed: 2022-11-28
- Published: 2025-03-20
- Earliest claimed: 2022-04-15
- Assignees: Suzhou Metabrain Intelligent Technology Co., Ltd.
- Applicants: Suzhou Metabrain Intelligent Technology Co., Ltd.
- Inventors: Hongli LIU; Feng LI; Tong YU; Chong SHEN
- Web page: https://patentgrep.com/patent/US20250094700A1

## CPC

- G06F 40/186: Templates
- G06F 40/30: Semantic analysis
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

The present application relates to the technical field of computers. Provided is a small sample fine-turning method, the method comprising: inputting a data set, and forming an input sample according to a fixed template; constructing a candidate tag word set and a candidate prompt template set; by means of reinforcement learning, searching an optimal tag word corresponding to the input sample from the candidate tag word set, and a prompt template corresponding to the input sample from the candidate prompt template set; and outputting a mapping relationship of the optimal tag word and an optimal prompt template format corresponding to the prompt template.

## Claims

### Claim 1

A small sample fine-tuning method, comprising:
inputting a data set, and forming an input sample according to a fixed template;
constructing a candidate tag word set and a candidate prompt template set;
by means of reinforcement learning, searching an optimal tag word corresponding to the input sample from the candidate tag word set, and a prompt template corresponding to the input sample from the candidate prompt template set; and
outputting a mapping relationship of the optimal tag word and an optimal prompt template format corresponding to the prompt template

### Claim 2 (depends on 1)

The small sample fine-tuning method according to claim 1, further comprising:
dividing the data set into a training set, a validation set, and a test set; wherein
the training set is configured to random sample to form the input sample; and
the validation set is configured to calculate a cosine similarity.

### Claim 3

The small sample fine-tuning method according to claim, further comprising:
forming the data in the data set according to ID attributes, sentence attributes, and label attributes, wherein, the ID attributes are configured to represent IDs of the data, the sentence attributes are configured to represent contents of the data, and the label attributes are configured to represent tag words of the data.

### Claim 4 (depends on 1)

The small sample fine-tuning method according to claim 1, wherein the step of inputting the data set, and forming the input sample according to the fixed template comprises:
acquiring input content;
representing the input content in the fixed template;
calculating a cosine similarity between the input content and all samples in a training set; and
random sampling from a preset percentage of training set samples to obtain the input sample.

### Claim 5 (depends on 4)

The small sample fine-tuning method according to claim 4, further comprising:
initializing a prompt template format;
representing the input content in the initialized prompt template format.

### Claim 6 (depends on 4)

The small sample fine-tuning method according to claim 4, wherein the step of calculating the cosine similarity between the input content and all samples in the training set comprises:
encoding the input content using the SBERT method; and
calculating, for each input content in a validation set, the cosine similarity to all samples in the training set respectively.

### Claim 7 (depends on 3)

The small sample fine-tuning method according to claim 3, further comprising:
converting the input sample to a prompts input.

### Claim 8 (depends on 1)

The small sample fine-tuning method according to claim 1, wherein the constructing the candidate tag word set and the candidate prompt template set comprises:
automatically selecting the optimal candidate tag word; and
automatically selecting a candidate prompt template.

### Claim 9 (depends on 8)

The small sample fine-tuning method according to claim 8, wherein the automatically selecting the candidate tag word comprises:
initializing a vocabulary;
vectorizing all the words in the vocabulary using a word2vec method, and determining a near-synonym set corresponding to each tag via the cosine similarity;
selecting, for each category in the training set, a word in the vocabulary that maximizes the conditional probability, and a conditional probability set comprising the word, by a pre-training model that is not fine-tuned:
determining a candidate tag word under each category as a maximum value of a geometric intersection of the near-synonym set and the conditional probability; and
integrating candidate tag words under various categories, and determining an assignment mode which maximizes the accuracy rate of the training set as the optimal candidate tag word.

### Claim 10 (depends on 9)

The small sample fine-tuning method according to claim 9, further comprising:
determining the conditional probability set through the formula:
Topk
v
∈
V
⁢
{
∑
x
in
∈
D
train
c
log
⁢
P
ℒ
(
[
MASK
]
=
v
⁢
❘
"\[LeftBracketingBar]"
T
⁡
(
x
in
)
)
}
;
wherein Topk is a word with the maximum conditional probability; V is an initialization vocabulary; is a pre-trained model that is not fine-tuned; c is each category in the training set; P represents the output probability distribution based on the model ; and T(Xin) is an input sample.

### Claim 11 (depends on 9)

The small sample fine-tuning method according to claim 9, wherein the automatically selecting the candidate prompt template comprises:
determining the optimal candidate tag word;
generating an initial prompt template by filling a placeholder; wherein the initial prompt template is configured to maximize an output probability in the training set; and
decoding the initial prompt template using a bundle search algorithm to obtain the candidate prompt template.

### Claim 12 (depends on 11)

The small sample fine-tuning method according to claim 11, wherein by means of reinforcement learning, searching the optimal tag word corresponding to the input sample from the candidate tag word set, and the prompt template corresponding to the input sample from the candidate prompt template set comprises:
determining a preset number of candidate tag word set for each category;
combining the candidate tag word set with a template set corresponding to the candidate prompt template to obtain a search space list;
by means of the search space list, determining an optimal tag word corresponding to the input sample from the candidate tag word set, and a prompt template corresponding to the input sample from the candidate prompt template set.

### Claim 13 (depends on 12)

The small sample fine-tuning method according to claim 12, further comprising:
by combining the candidate tag word set with a template set corresponding to the candidate prompt template, obtaining the search space list, to determine the optimal assignment mode of the candidate tag word and the candidate prompt template in the fine-tuning process.

### Claim 14 (depends on 1)

The small sample fine-tuning method according to claim 1, further comprising:
determining the optimal tag word and the prompt template by key factors in reinforcement learning, wherein the key factors comprise agent, environment, action, status, and reward.

### Claim 15 (depends on 14)

The small sample fine-tuning method according to claim 14, wherein the step of determining the optimal tag word and the optimal prompt template format comprises:
inputting the text into the model to obtain an output result; the model comprising a language model environment;
calculating a loss of the output result and the tag word;
feeding back the loss as the reward to the agent; and
determining, by the agent, selection directions of the template and the tag word according to the reward until the optimal tag word and the prompt template are determined.

### Claim 16 (depends on 1)

The small sample fine-tuning method according to claim 1, further comprising:
when the input is textless, averaging the output tag word corresponding probability and then normalizing to obtain a normalized probability p_cf; and calculating a correction matrix according to the formula [diag(p_cf)]−1.

### Claim at position 17

17-18. (canceled)

### Claim 19 (depends on 1)

A non-volatile readable storage medium having stored thereon a computer program that, when executed by a processor, implements the steps of the method according to claim 1.

### Claim 20 (depends on 1)

An electronic device, comprising a memory having stored thereon a computer program, and a processor that implements the steps of the method according to claim 1 when calling the computer program in the memory.

### Claim 21 (depends on 20)

The electronic device according to claim 20, wherein the constructing the candidate tag word set and the candidate prompt template set comprises:
automatically selecting the optimal candidate tag word; and
automatically selecting a candidate prompt template.

### Claim 22 (depends on 21)

The electronic device according to claim 21, wherein the automatically selecting the candidate tag word comprises:
initializing a vocabulary;
vectorizing all the words in the vocabulary using a word2vec method, and determining a near-synonym set corresponding to each tag via the cosine similarity;
selecting, for each category in the training set, a word in the vocabulary that maximizes the conditional probability, and a conditional probability set comprising the word, by a pre-training model that is not fine-tuned:
determining a candidate tag word under each category as a maximum value of a geometric intersection of the near-synonym set and the conditional probability; and
integrating candidate tag words under various categories, and determining an assignment mode which maximizes the accuracy rate of the training set as the optimal candidate tag word.
