# US 12,602,410 B2 · Multimodal context selection for large language model based resolutions addressing technical issues > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,602,410 B2 - Document ID: US12602410B2 - Status: Granted - Kind: B2 - Application number: 18/631,590 - Filed: 2024-04-10 - Published: 2025-10-16 - Granted: 2026-04-14 - Earliest claimed: 2024-04-10 - Assignees: Dell Products L.P. - Applicants: Dell Products L.P. - Inventors: Ravi Shukla; Gaurav Bhattacharjee; Ramakanth Kanagovi - Web page: https://patentgrep.com/patent/US12602410B2 ## CPC - G06F 16/3329: Natural language query formulation - G06F 16/3344: {using natural language analysis} - G06F 16/383: using metadata automatically derived from the content - G06F 16/93: Document management systems - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A method for technical issue resolution. The method includes: receiving, from a user, a text query concerning a technical issue; obtaining query-related context relevant to the text query; and processing, through a large language model (LLM), the text query and the query-related context to produce a multimodal query response used by the user to address the technical issue. More specifically, embodiments described herein utilize text topic and zero shot classification models to translate multimodal technical documentation (e.g., including text and images) into topic relevant metadata; and process queries, pertaining to technical issues, using a multimodal LLM provided with query-related text and image context derived from said topic relevant metadata. ## Claims ### Claim 1 A method for technical issue resolution, the method comprising: extracting a text portion of a multimodal document; partitioning the text portion into text portion chunks; processing, using a text topic model, the text portion chunks to discover a plurality of text topics associated with the multimodal document and to obtain a normalized topic weight between each text portion chunk and each text topic; selecting, for each text topic, a text portion chunks subset comprising a plurality of text portion chunks each mapped to the text topic and to the normalized topic weight at least meeting a normalized topic weight threshold; and combining, for each text topic, the text portion chunks subset to obtain a topic-related text representative of the text topic; receiving, from a user and following combining the text portion chunks subset, a text query concerning a technical issue; obtaining query-related context relevant to the text query, wherein obtaining the query-related context, comprises: identifying a query-related text topic of the topic-related text for the text query; selecting at least one query-related text topic chunk each representing a text portion chunk mapped to the query-related text topic, wherein the text portion chunk is further mapped to the normalized topic weight at least equal to the normalized topic weight threshold; combining the at least one query-related text topic chunk to obtain query-related text context; selecting at least one query-related image subset each representing an image portion subset mapped to the query-related text topic, wherein the image portion subset is further mapped to an image-topic similarity score at least equal to an image-topic similarity score threshold; and combining the at least one query-related image subset and the query-related text context to obtain the query-related context; and processing, through a large language model (LLM), the text query and the query-related context to produce a multimodal query response used by the user to address the technical issue. ### Claim 2 (depends on 1) The method of claim 1, wherein the text portion chunk and the image portion subset belong to a same multimodal document. ### Claim 3 (depends on 1) The method of claim 1, wherein the text portion chunk and the image portion subset belong to different multimodal documents. ### Claim 4 (depends on 1) The method of claim 1, wherein identifying the query-related text topic, comprises: obtaining topic-related texts representative of a plurality of text topics; processing, using a zero shot text model, the text query and the topic-related texts to produce a query-topic vector comprising a query-topic similarity score for each topic-related text; and identifying a text topic mapped to a topic-related text mapped to a highest query-topic similarity score as the query-related text topic. ### Claim 5 (depends on 1) The method of claim 1, wherein the multimodal query response comprises text content and image content. ### Claim 6 (depends on 1) The method of claim 1, the method further comprising: extracting, from the multimodal document, an image portion comprising at least one image; and processing, using a zero shot image model, the at least one image and the plurality of text topics to obtain an image-topic similarity score between each image and each text topic. ### Claim 7 A non-transitory computer readable medium (CRM) comprising computer readable program code, which when executed by a computer processor, enables the computer processor to perform a method for technical issue resolution, the method comprising: extracting a text portion of a multimodal document; partitioning the text portion into text portion chunks; processing, using a text topic model, the text portion chunks to discover a plurality of text topics associated with the multimodal document and to obtain a normalized topic weight between each text portion chunk and each text topic; selecting, for each text topic, a text portion chunks subset comprising a plurality of text portion chunks each mapped to the text topic and to the normalized topic weight at least meeting a normalized topic weight threshold; and combining, for each text topic, the text portion chunks subset to obtain a topic-related text representative of the text topic; receiving, from a user and following combining the text portion chunks subset, a text query concerning a technical issue; obtaining query-related context relevant to the text query, wherein obtaining the query-related context, comprises: identifying a query-related text topic of the topic-related text for the text query; selecting at least one query-related text topic chunk each representing a text portion chunk mapped to the query-related text topic, wherein the text portion chunk is further mapped to the normalized topic weight at least equal to the normalized topic weight threshold; combining the at least one query-related text topic chunk to obtain query-related text context; selecting at least one query-related image subset each representing an image portion subset mapped to the query-related text topic, wherein the image portion subset is further mapped to an image-topic similarity score at least equal to an image-topic similarity score threshold; and combining the at least one query-related image subset and the query-related text context to obtain the query-related context; and processing, through a large language model (LLM), the text query and the query-related context to produce a multimodal query response used by the user to address the technical issue. ### Claim 8 (depends on 7) The non-transitory CRM of claim 7, wherein the text portion chunk and the image portion subset belong to a same multimodal document. ### Claim 9 (depends on 7) The non-transitory CRM of claim 7, wherein the text portion chunk and the image portion subset belong to different multimodal documents. ### Claim 10 (depends on 7) The non-transitory CRM of claim 7, wherein identifying the query-related text topic, comprises: obtaining topic-related texts representative of a plurality of text topics; processing, using a zero shot text model, the text query and the topic-related texts to produce a query-topic vector comprising a query-topic similarity score for each topic-related text; and identifying a text topic mapped to a topic-related text mapped to a highest query-topic similarity score as the query-related text topic. ### Claim 11 (depends on 7) The non-transitory CRM of claim 7, wherein the multimodal query response comprises text content and image content. ### Claim 12 (depends on 7) The non-transitory CRM of claim 7, the method further comprising: extracting, from the multimodal document, an image portion comprising at least one image; and processing, using a zero shot image model, the at least one image and the plurality of text topics to obtain an image-topic similarity score between each image and each text topic. ### Claim 13 A system, comprising: a large language model (LLM); and a multimodal context selector operatively connected to the LLM, and comprising a computer processor configured to perform a method for technical issue resolution, the method comprising: extracting a text portion of a multimodal document; partitioning the text portion into text portion chunks; processing, using a text topic model, the text portion chunks to discover a plurality of text topics associated with the multimodal document and to obtain a normalized topic weight between each text portion chunk and each text topic; selecting, for each text topic, a text portion chunks subset comprising a plurality of text portion chunks each mapped to the text topic and to the normalized topic weight at least meeting a normalized topic weight threshold; and combining, for each text topic, the text portion chunks subset to obtain a topic-related text representative of the text topic; receiving, from a user and following combining the text portion chunks subset, a text query concerning a technical issue; obtaining query-related context relevant to the text query, wherein obtaining the query-related context, comprises: identifying a query-related text topic of the topic-related text for the text query; selecting at least one query-related text topic chunk each representing a text portion chunk mapped to the query-related text topic, wherein the text portion chunk is further mapped to the normalized topic weight at least equal to the normalized topic weight threshold; combining the at least one query-related text topic chunk to obtain query-related text context; selecting at least one query-related image subset each representing an image portion subset mapped to the query-related text topic, wherein the image portion subset is further mapped to an image-topic similarity score at least equal to an image-topic similarity score threshold; and combining the at least one query-related image subset and the query-related text context to obtain the query-related context; and processing, through the LLM, the text query and the query-related context to produce a multimodal query response used by the user to address the technical issue. ### Claim 14 (depends on 13) The system of claim 13, wherein the multimodal query response comprises text content and image content.