Multimodal context selection for large language model based resolutions addressing technical issues

A non-transitory computer readable medium (CRM) comprising computer readable program code, which when executed by a computer processor, enables the computer processor to perform a method for technical issue resolution, the method comprising:

• extracting a text portion of a multimodal document;
• partitioning the text portion into text portion chunks;
• processing, using a text topic model, the text portion chunks to discover a plurality of text topics associated with the multimodal document and to obtain a normalized topic weight between each text portion chunk and each text topic;
• selecting, for each text topic, a text portion chunks subset comprising a plurality of text portion chunks each mapped to the text topic and to the normalized topic weight at least meeting a normalized topic weight threshold;
• and
• combining, for each text topic, the text portion chunks subset to obtain a topic-related text representative of the text topic;
• receiving, from a user and following combining the text portion chunks subset, a text query concerning a technical issue;
• obtaining query-related context relevant to the text query, wherein obtaining the query-related context, comprises:
• identifying a query-related text topic of the topic-related text for the text query;
• selecting at least one query-related text topic chunk each representing a text portion chunk mapped to the query-related text topic, wherein the text portion chunk is further mapped to the normalized topic weight at least equal to the normalized topic weight threshold;
• combining the at least one query-related text topic chunk to obtain query-related text context;
• selecting at least one query-related image subset each representing an image portion subset mapped to the query-related text topic, wherein the image portion subset is further mapped to an image-topic similarity score at least equal to an image-topic similarity score threshold;
• and
• combining the at least one query-related image subset and the query-related text context to obtain the query-related context;
• and
• processing, through a large language model (LLM), the text query and the query-related context to produce a multimodal query response used by the user to address the technical issue.

A system, comprising:

• a large language model (LLM);
• and
• a multimodal context selector operatively connected to the LLM, and comprising a computer processor configured to perform a method for technical issue resolution, the method comprising:
• extracting a text portion of a multimodal document;
• partitioning the text portion into text portion chunks;
• processing, using a text topic model, the text portion chunks to discover a plurality of text topics associated with the multimodal document and to obtain a normalized topic weight between each text portion chunk and each text topic;
• selecting, for each text topic, a text portion chunks subset comprising a plurality of text portion chunks each mapped to the text topic and to the normalized topic weight at least meeting a normalized topic weight threshold;
• and
• combining, for each text topic, the text portion chunks subset to obtain a topic-related text representative of the text topic;
• receiving, from a user and following combining the text portion chunks subset, a text query concerning a technical issue;
• obtaining query-related context relevant to the text query, wherein obtaining the query-related context, comprises:
• identifying a query-related text topic of the topic-related text for the text query;
• selecting at least one query-related text topic chunk each representing a text portion chunk mapped to the query-related text topic, wherein the text portion chunk is further mapped to the normalized topic weight at least equal to the normalized topic weight threshold;
• combining the at least one query-related text topic chunk to obtain query-related text context;
• selecting at least one query-related image subset each representing an image portion subset mapped to the query-related text topic, wherein the image portion subset is further mapped to an image-topic similarity score at least equal to an image-topic similarity score threshold;
• and
• combining the at least one query-related image subset and the query-related text context to obtain the query-related context;
• and
• processing, through the LLM, the text query and the query-related context to produce a multimodal query response used by the user to address the technical issue.