Multi-modal prompt generation for generative artificial intelligence

A non-transitory computer-readable medium storing instructions which, when executed by a processing system including at least one processor, cause the processing system to perform operations, the operations comprising:

• identifying a user of a generative artificial intelligence model;
• detecting an input from the user that triggers a query to the generative artificial intelligence model, wherein the input is received via at least one modality other than a text modality or an image modality;
• constructing a prompt for the generative artificial intelligence model based on the input;
• executing the generative artificial intelligence model, using the prompt as an input to the generative artificial intelligence model, to generate an output;
• and
• delivering the output to a user endpoint device of the user.

A device comprising:

• a processing system including at least one processor;
• and
• a non-transitory computer-readable medium storing instructions which, when executed by the processing system, cause the processing system to perform operations, the operations comprising:
• identifying a user of a generative artificial intelligence model;
• detecting an input from the user that triggers a query to the generative artificial intelligence model, wherein the input is received via at least one modality other than a text modality or an image modality;
• constructing a prompt for the generative artificial intelligence model based on the input;
• executing the generative artificial intelligence model, using the prompt as an input to the generative artificial intelligence model, to generate an output;
• and
• delivering the output to a user endpoint device of the user.