Multimodal data processing

An electronic device, comprising:

• at least one processing unit;
• and
• at least one memory coupled to the at least one processing unit and storing instructions executable by the at least one processing unit, the instructions, upon execution by the at least one processing unit, causing the device to perform:
• obtaining image data and text data;
• and
• extracting, using a feature extraction model, a target visual feature of the image data and a target textual feature of the text data, the feature extraction model comprises alternately deployed cross-modal encoding parts and visual encoding parts, and the extracting comprises:
• performing, using a first cross-modal encoding part of the feature extraction model, cross-modal feature encoding on a first intermediate visual feature of the image data and a first intermediate textual feature of the text data, to obtain a second intermediate visual feature and a second intermediate textual feature;
• performing, using a first visual encoding part of the feature extraction model, visual modal feature encoding on the second intermediate visual feature, to obtain a third intermediate visual feature;
• performing, using a second cross-modal encoding part of the feature extraction model, cross-modal feature encoding on the third intermediate visual feature and the second intermediate textual feature, to obtain a fourth intermediate visual feature and a third intermediate textual feature;
• and
• determining the target visual feature and the target textual feature based on the fourth intermediate visual feature and the third intermediate textual feature.

A non-transitory computer-readable storage medium, having a computer program stored thereon which, upon execution by a processor, causes the following acts to be performed:

• obtaining image data and text data;
• and
• extracting, using a feature extraction model, a target visual feature of the image data and a target textual feature of the text data, the feature extraction model comprises alternately deployed cross-modal encoding parts and visual encoding parts, and the extracting comprises:
• performing, using a first cross-modal encoding part of the feature extraction model, cross-modal feature encoding on a first intermediate visual feature of the image data and a first intermediate textual feature of the text data, to obtain a second intermediate visual feature and a second intermediate textual feature;
• performing, using a first visual encoding part of the feature extraction model, visual modal feature encoding on the second intermediate visual feature, to obtain a third intermediate visual feature;
• performing, using a second cross-modal encoding part of the feature extraction model, cross-modal feature encoding on the third intermediate visual feature and the second intermediate textual feature, to obtain a fourth intermediate visual feature and a third intermediate textual feature;
• and
• determining the target visual feature and the target textual feature based on the fourth intermediate visual feature and the third intermediate textual feature.