Methods and apparatuses for training content understanding model and content generation model

A non-transitory, computer-readable medium storing one or more instructions executable by a computer system to perform one or more operations, comprising:

• separately training a content understanding model and a content generation model with an image-text pair formed by an image and a text in a target training set, wherein the content understanding model is used to generate an image description text based on an input image, and wherein the content generation model is used to generate a corresponding image based on an input description text;
• and
• performing sample processing on a noise-containing sample set, wherein an image-text matching degree of an image-text pair in the noise-containing sample set is less than that in the target training set, and the sample processing comprises:
• inputting a first image in any first image-text pair in the noise-containing sample set into the content understanding model to obtain several candidate texts;
• separately inputting a first text in the first image-text pair and the several candidate texts into the content generation model to obtain multiple candidate images;
• performing similarity matching between the multiple candidate images and the first image, and determining a target text based on a matching result;
• and
• forming a second image-text pair by using the first image and the target text and adding the second image-text pair to the target training set, as an updated target training set, to continue to train the content understanding model and the content generation model.

A computer-implemented system, comprising:

• one or more computers;
• and
• one or more computer memory devices interoperably coupled with the one or more computers and having tangible, non-transitory, machine-readable media storing one or more instructions that, when executed by the one or more computers, perform one or more operations, comprising:
• separately training a content understanding model and a content generation model with an image-text pair formed by an image and a text in a target training set, wherein the content understanding model is used to generate an image description text based on an input image, and wherein the content generation model is used to generate a corresponding image based on an input description text;
• and
• performing sample processing on a noise-containing sample set, wherein an image-text matching degree of an image-text pair in the noise-containing sample set is less than that in the target training set, and the sample processing comprises:
• inputting a first image in any first image-text pair in the noise-containing sample set into the content understanding model to obtain several candidate texts;
• separately inputting a first text in the first image-text pair and the several candidate texts into the content generation model to obtain multiple candidate images;
• performing similarity matching between the multiple candidate images and the first image, and determining a target text based on a matching result;
• and
• forming a second image-text pair by using the first image and the target text and adding the second image-text pair to the target training set, as an updated target training set, to continue to train the content understanding model and the content generation model.