Multi-modal fusion data processing method, apparatus, medium, electronic device, and program product

A non-transitory computer-readable medium having a computer program stored thereon, wherein the computer program which, when executed by a processing apparatus, causes the processing apparatus to:

• receive a multi-modal data retrieval statement, parse the multi-modal data retrieval statement, and translate the multi-modal data retrieval statement into single-modal data retrieval statements respectively corresponding to a plurality of modalities;
• perform, according to each of the single-modal data retrieval statements, data retrieval in a corresponding single-modal database in a multi-modal database to obtain target data of different modalities, wherein single-modal databases respectively corresponding to the plurality of modalities are maintained in the multi-modal database, and data management is performed independently for each of the single-modal databases;
• and
• perform feature fusion and modality alignment processing on the retrieved target data of different modalities to generate a unified multi-modal data representation, and feed back the unified multi-modal data representation as a retrieval result.

An electronic device, comprising:

• a storage apparatus having a computer program stored thereon;
• and
• a processing apparatus, configured to execute the computer program in the storage apparatus to:
• receive a multi-modal data retrieval statement, parse the multi-modal data retrieval statement, and translate the multi-modal data retrieval statement into single-modal data retrieval statements respectively corresponding to a plurality of modalities;
• perform, according to each of the single-modal data retrieval statements, data retrieval in a corresponding single-modal database in a multi-modal database to obtain target data of different modalities, wherein single-modal databases respectively corresponding to the plurality of modalities are maintained in the multi-modal database, and data management is performed independently for each of the single-modal databases;
• and
• perform feature fusion and modality alignment processing on the retrieved target data of different modalities to generate a unified multi-modal data representation, and feed back the unified multi-modal data representation as a retrieval result.