Dual Attention Network Using Transformers for Cross-Modal Retrieval
OwnerHamid R. Tizhoosh, Danial Malekiinventors
Life of this patent
Estimating the term…
Abstract
Cross-modal data are extracted from an input data source. A computer system accesses first input data of a first modality and second input data of a second modality. The first and second input data are then input to a dual attention network that has been trained on training data to extract feature data from cross-modal input data. Feature data are generated as outputs by inputting the first and second input data to the dual attention network. The feature data include feature representations of the first modality and the second modality. The feature data can be stored or displayed to a user with the computer system.
Claims20
A method for extracting features from cross-modal data from an input data source, the method comprising:
- (a) accessing, with a computer system, first input data of a first modality;
- (b) accessing, with the computer system, second input data of a second modality;
- (c) accessing, with the computer system, a dual attention network trained on training data to extract feature data from cross-modal input data;
- (d) inputting the first input data and the second input data to the dual attention network by the computer system, generating outputs as feature data comprising feature representations of the first modality and the second modality;
- and
- (e) storing the feature data or displaying the feature data to a user with the computer system.
Dependencies are inferred from the claim text.