Efficient Hybrid Generative AI via Context Filtering/Focused Attention

A method performed by a processor of an end-user computing device of applying multimodal data to an artificial intelligence (AI) model, the method comprising:

• receiving multimodal data;
• determining user intent based on information available to the processor;
• generating filtered input data by performing context filtering on the multimodal data based on the determined user intent to generate filtered input data;
• generating filtered data segments by segmenting the filtered input data based on the determined user intent;
• converting the filtered data segments into tokens representing attributes of the data segments;
• assigning a priority to each of the tokens based on their relevance to the determined user intent;
• generating an enhanced prompt based on the assigned token priorities;
• sending the enhanced prompt to an AI model;
• receiving inference results from the AI model;
• generating a final output based on the received inference results and locally processed data;
• and
• presenting the final output to a user.

A non-transitory processor-readable medium having stored thereon processor-readable instructions configured to cause a processor of a computing device to perform operations comprising:

• receiving multimodal data;
• determining user intent based on information available to the processor;
• generating filtered input data by performing context filtering on the multimodal data based on the determined user intent to generate filtered input data;
• generating filtered data segments by segmenting the filtered input data based on the determined user intent;
• converting the filtered data segments into tokens representing attributes of the data segments;
• assigning a priority to each of the tokens based on their relevance to the determined user intent;
• generating an enhanced prompt based on the assigned token priorities;
• sending the enhanced prompt to a local or remote artificial intelligence (AI) model;
• receiving inference results from the AI model;
• generating a final output based on the received inference results and locally processed data;
• and
• presenting the final output to a user.