Data processing method and apparatus

A data processing apparatus comprising:

• at least one processor configured to execute a computer program stored in a memory such that the data processing apparatus is configured to:
• determine original text samples, wherein masking processing has not been previously performed on the original text samples;
• perform mask processing on the original text samples to obtain mask training samples, wherein the mask processing makes mask proportions of the mask training samples unfixed;
• and
• train a pretrained language model (PLM) using the mask training samples;
• wherein the data processing apparatus is further configured to:
• obtain a word-level mask proportion of each word in a first text sample in the original text samples, wherein word-level mask proportions of different words in the first text sample are not exactly the same;
• and
• perform masking processing on some words in the first text sample based on the word-level mask proportion of each word in the first text sample, to obtain a first training sample in the mask training samples;
• wherein the data processing apparatus is further configured to input the first text sample into a neural network model, and obtain the word-level mask proportion of each word in the first text sample from an output of the neural network model, wherein
• the neural network model is obtained through optimization learning by using the following steps, and an initial value of i is 1:
• (a) input a training sample corresponding to an ith sample in the original text samples into the PLM, to obtain loss values output by the PLM for masked words;
• and
• (b) update and optimize a neural network model based on the loss values output by the PLM for the masked words and output signals of the neural network model for the masked words.

A chip comprising at least one processor and a data interface, wherein the at least one processor is configured to invoke and run, through the data interface, a computer program stored in a memory, to enable the chip to at least be configured to:

• determine original text samples, wherein masking processing has not been previously performed on the original text samples;
• perform mask processing on the original text samples to obtain mask training samples, wherein the mask processing makes mask proportions of the mask training samples unfixed;
• and
• train a pretrained language model (PLM) using the mask training samples;
• wherein the chip is configured to perform the mask processing on the original text samples to obtain the mask training samples by being configured to:
• obtain a word-level mask proportion of each word in a first text sample in the original text samples, wherein word-level mask proportions of different words in the first text sample are not exactly the same;
• and
• perform masking processing on some words in the first text sample based on the word-level mask proportion of each word in the first text sample, to obtain a first training sample in the mask training samples;
• and
• wherein the chip is configured to obtain the word-level mask proportion of each word in a first text sample in the original text samples by being configured to:
• input the first text sample into a neural network model, and obtaining the word-level mask proportion of each word in the first text sample from an output of the neural network model, wherein the neural network model is obtained through optimization learning by using the following steps, and an initial value of i is 1:
• (a) input a training sample corresponding to an ith sample in the original text samples into the PLM, to obtain loss values of the PLM for masked words;
• and
• (b) update and optimize a neural network model based on the loss values output by the PLM for the masked words and output signals of the neural network model for the masked words.