Efficiency and flexibility of machine learning models

A system of improving efficiency and flexibility of a machine learning model, comprising:

• at least one processor;
• and
• at least one memory communicatively coupled to the at least one processor and comprising computer-readable instructions that upon execution by the at least one processor cause the at least one processor to perform operations comprising:
• configuring a machine learning model by decomposing self-attention in the machine learning model into a plurality of attention operations, wherein the machine learning model is configured to process information from a plurality of modalities;
• receiving concatenated tokens by the machine learning model, wherein the concatenated tokens comprise multimodal tokens representative of a content item and textual tokens indicative of a text query;
• generating updated multimodal tokens for a next layer of computation by performing diagonal-attention on the multimodal tokens;
• and
• generating updated textual tokens for the next layer of computation by performing self-attention on the textual tokens and performing cross-attention between the multimodal tokens and the textual tokens.

A non-transitory computer-readable storage medium, storing computer-readable instructions that upon execution by a processor cause the processor to implement operations comprising:

• configuring a machine learning model by decomposing self-attention in the machine learning model into a plurality of attention operations, wherein the machine learning model is configured to process information from a plurality of modalities;
• receiving concatenated tokens by the machine learning model, wherein the concatenated tokens comprise multimodal tokens representative of a content item and textual tokens indicative of a text query;
• generating updated multimodal tokens for a next layer of computation by performing diagonal-attention on the multimodal tokens;
• and
• generating updated textual tokens for the next layer of computation by performing self-attention on the textual tokens and performing cross-attention between the multimodal tokens and the textual tokens.