Text processing model training method, and text processing method and apparatus

A text processing method performed by a processor, comprising:

• obtaining to-be-processed text;
• and
• processing the to-be-processed text using a target student model to obtain a processing result of the to-be-processed text, wherein:
• the target student model is obtained by training a student model based on sample data and prediction data, the sample data is obtained based on training text and a teacher model, and the prediction data is obtained based on the training text and the student model;
• the teacher model and the student model each comprise an input layer, one or more intermediate layers, and an output layer;
• the sample data comprises a sample semantic feature output by the intermediate layer of the teacher model and a sample label output by the output layer of the teacher model;
• the prediction data comprises a prediction semantic feature output by the intermediate layer of the student model and a prediction label output by the output layer of the student model;
• and the teacher model is a pre-trained model used for text processing,
• the training text is an enhanced text set, each piece of training text in the enhanced text set comprises label information, and the label information comprises any one of information indicating a classification result of the training text, sequence tagging information, or text translation label information;
• the classification result comprises an emotion classification result of the training text or a semantic intention classification result of the training text;
• and the enhanced text set is obtained based on an obtained original text set and a replacement text set, and the replacement text set is obtained by performing partial word replacement on the original text set, and
• the replacement text set comprises replacement text of first text, the first text is any text in the original text set, and the replacement text of the first text is obtained by inputting masked text into a pre-trained language model;
• and the masked text is obtained by masking one or more words at a replacement position in the first text, and the replacement position is determined based on a replacement threshold of each word in the first text.

A text processing apparatus, comprising:

• a memory, configured to store a program;
• and
• a processor, configured to execute the program stored in the memory, wherein when the processor executes the program stored in the memory, the processor is configured to perform:
• obtaining to-be-processed text;
• and
• processing the to-be-processed text using a target student model to obtain a processing result of the to-be-processed text, wherein
• the target student model is obtained by training a student model based on sample data and prediction data, the sample data is obtained based on training text and a teacher model, and the prediction data is obtained based on the training text and the student model;
• the teacher model and the student model each comprise an input layer, one or more intermediate layers, and an output layer;
• the sample data comprises a sample semantic feature output by the intermediate layer of the teacher model and a sample label output by the output layer of the teacher model;
• the prediction data comprises a prediction semantic feature output by the intermediate layer of the student model and a prediction label output by the output layer of the student model;
• and the teacher model is a pre-trained model used for text processing,
• the training text is an enhanced text set, each piece of training text in the enhanced text set comprises label information, and the label information comprises any one of information indicating a classification result of the training text, sequence tagging information, or text translation label information;
• the classification result comprises an emotion classification result of the training text or a semantic intention classification result of the training text;
• and the enhanced text set is obtained based on an obtained original text set and a replacement text set, and the replacement text set is obtained by performing partial word replacement on the original text set, and
• the replacement text set comprises replacement text of first text, the first text is any text in the original text set, and the replacement text of the first text is obtained by inputting masked text into a pre-trained language model;
• and the masked text is obtained by masking one or more words at a replacement position in the first text, and the replacement position is determined based on a replacement threshold of each word in the first text.

A non-transitory computer-readable storage medium, wherein the computer-readable storage medium stores program instructions, and when the program instructions are run on a processor, the processor is configured to perform:

• obtaining to-be-processed text;
• and
• processing the to-be-processed text using a target student model to obtain a processing result of the to-be-processed text, wherein
• the target student model is obtained by training a student model based on sample data and prediction data, the sample data is obtained based on training text and a teacher model, and the prediction data is obtained based on the training text and the student model;
• the teacher model and the student model each comprise an input layer, one or more intermediate layers, and an output layer;
• the sample data comprises a sample semantic feature output by the intermediate layer of the teacher model and a sample label output by the output layer of the teacher model;
• the prediction data comprises a prediction semantic feature output by the intermediate layer of the student model and a prediction label output by the output layer of the student model;
• and the teacher model is a pre-trained model used for text processing,
• the training text is an enhanced text set, each piece of training text in the enhanced text set comprises label information, and the label information comprises any one of information indicating a classification result of the training text, sequence tagging information, or text translation label information;
• the classification result comprises an emotion classification result of the training text or a semantic intention classification result of the training text;
• and the enhanced text set is obtained based on an obtained original text set and a replacement text set, and the replacement text set is obtained by performing partial word replacement on the original text set, and
• the replacement text set comprises replacement text of first text, the first text is any text in the original text set, and the replacement text of the first text is obtained by inputting masked text into a pre-trained language model;
• and the masked text is obtained by masking one or more words at a replacement position in the first text, and the replacement position is determined based on a replacement threshold of each word in the first text.