Large model-based method of generating text and method of training text generation model

A method of training a text generation model, comprising:

• acquiring a sample text, wherein the sample text comprises a plurality of text segments obtained by dividing a long text;
• for each text segment of the plurality of text segments:
• acquiring a memory state for the text segment, wherein the memory state is generated based on a previous text of the text segment;
• determining an embedding feature of the text segment as an initial hidden state, and processing the memory state and the initial hidden state by using an encoding network in the text generation model to obtain an updated hidden state;
• and
• generating, based on the updated hidden state, a subsequent text for the text segment by using an output layer in the text generation model;
• and
• training the text generation model based on the sample text and subsequent texts for the plurality of text segments,
• wherein the encoding network is configured to process the memory state and the initial hidden state by using a first attention mechanism.

An electronic device, comprising:

• at least one processor;
• and
• a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions, when executed by the at least one processor, are configured to cause the at least one processor to at least:
• acquire a memory state for a text to be processed, wherein the memory state is generated based on a previous text of the text to be processed;
• determine an embedding feature of the text to be processed as an initial hidden state, and process the memory state and the initial hidden state by using a first attention mechanism to obtain an updated hidden state;
• and
• generate a subsequent text for the text to be processed based on the updated hidden state.