# US 12,762,686 B2 · Machine-learning based defect identification in images of computing devices > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,762,686 B2 - Document ID: US12762686B2 - Status: Granted - Kind: B2 - Application number: 19/002,061 - Filed: 2024-12-26 - Published: 2026-07-02 - Granted: 2026-10-13 - Earliest claimed: 2024-12-26 - Assignees: Dell Products L.P. - Applicants: Dell Products L.P. - Inventors: Jeffrey Scott Vah; Ravi Shukla; Aditi Saluja; Cachelle Johnson-Lewis - Web page: https://patentgrep.com/patent/US12762686B2 ## CPC - G06T 7/0004: {Industrial image inspection} - G06T 2207/20081: Training; Learning - G06T 2207/30141: Printed circuit board [PCB] - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract An apparatus comprises a processing device configured to obtain an image of a computing device, to generate a first data structure by processing the image to generate two or more image segments, to determine labels associated with designated defect types to be detected, and to apply the labels and the first data structure to a first machine learning model to generate a second data structure characterizing probabilities of the designated defect types being present in the image segments. The at least one processing device is further configured to determine input prompts based on the second data structure, and to apply the input prompts and the second data structure to a second machine learning model to generate a third data structure characterizing answers to the input prompts, and to identify, based on the third data structure, whether the computing device includes any defects of the designated defect types. ## Claims ### Claim 1 An apparatus comprising: at least one processing device comprising a processor coupled to a memory; the at least one processing device being configured: to obtain an image of at least a portion of a computing device; to process the image of the portion of the computing device to generate two or more segments thereof for inclusion in a first data structure; to determine a set of one or more labels to be provided as input to a first machine learning model, the one or more labels being associated with one or more designated defect types to be detected; to apply the determined set of one or more labels and at least a portion of the first data structure to the first machine learning model to determine, for inclusion in a second data structure, probabilities of the one or more designated defect types being present in each of the two or more segments of the image of the portion of the computing device; to determine, based at least in part on at least a portion of the second data structure, one or more input prompts for a second machine learning model; to apply the determined one or more input prompts and at least a portion of the second data structure to the second machine learning model to generate answers to the determined one or more input prompts; and to identify, based at least in part on the generated answers to the determined one or more input prompts, whether the portion of the computing device includes any defects of the one or more designated defect types. ### Claim 2 (depends on 1) The apparatus of claim 1 wherein the portion of the computing device comprises one or more hardware components of the computing device. ### Claim 3 (depends on 1) The apparatus of claim 1 wherein the portion of the computing device comprises a printed circuit board. ### Claim 4 (depends on 3) The apparatus of claim 3 wherein the one or more designated defect types comprises scratches on components of the printed circuit board. ### Claim 5 (depends on 3) The apparatus of claim 3 wherein the one or more designated defect types comprises burn marks on components of the printed circuit board. ### Claim 6 (depends on 3) The apparatus of claim 3 wherein the one or more designated defect types comprises missing components on the printed circuit board. ### Claim 7 (depends on 1) The apparatus of claim 1 wherein a segment size of the two or more segments of the image of the portion of the computing device is dynamically selected based at least in part on the one or more designated defect types to be detected. ### Claim 8 (depends on 1) The apparatus of claim 1 wherein the first machine learning model comprises a Contrastive Language-Image Pre-training (CLIP) model. ### Claim 9 (depends on 8) The apparatus of claim 8 wherein the CLIP model utilizes zero-shot learning with the determined set of one or more labels. ### Claim 10 (depends on 8) The apparatus of claim 8 wherein the CLIP model is fine-tuned utilizing a database of images of portions of computing devices having defects of the one or more designated defect types. ### Claim 11 (depends on 1) The apparatus of claim 1 wherein determining the one or more input prompts comprises selecting, from a set of possible input prompts, a subset of the set of possible input prompts which are associated with at least one of the one or more designated defect types having at least a threshold probability of being present in at least one of the two or more segments of the image of the portion of the computing device. ### Claim 12 (depends on 11) The apparatus of claim 11 wherein the selected subset of the set of possible input prompts comprises at least one input prompt associated with determining locations of one or more defects of the at least one designated defect type having at least the threshold probability of being present in at least one of the two or more segments of the image of the portion of the computing device. ### Claim 13 (depends on 11) The apparatus of claim 11 wherein the selected subset of the set of possible input prompts comprises at least one input prompt associated with determining a count of defects of the at least one designated defect type having at least the threshold probability of being present in at least one of the two or more segments of the image of the portion of the computing device. ### Claim 14 (depends on 1) The apparatus of claim 1 wherein the second machine learning model comprises a Visual Question Answering (VQA) model. ### Claim 15 A computer program product comprising a non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code when executed by at least one processing device causes the at least one processing device: to obtain an image of at least a portion of a computing device; to process the image of the portion of the computing device to generate two or more segments thereof for inclusion in a first data structure; to determine a set of one or more labels to be provided as input to a first machine learning model, the one or more labels being associated with one or more designated defect types to be detected; to apply the determined set of one or more labels and at least a portion of the first data structure to the first machine learning model to determine, for inclusion in a second data structure, probabilities of the one or more designated defect types being present in each of the two or more segments of the image of the portion of the computing device; to determine, based at least in part on at least a portion of the second data structure, one or more input prompts for a second machine learning model; to apply the determined one or more input prompts and at least a portion of the second data structure to the second machine learning model to generate answers to the determined one or more input prompts; and to identify, based at least in part on the generated answers to the determined one or more input prompts, whether the portion of the computing device includes any defects of the one or more designated defect types. ### Claim 16 (depends on 15) The computer program product of claim 15 wherein a segment size of the two or more segments of the image of the portion of the computing device is dynamically selected based at least in part on the one or more designated defect types to be detected. ### Claim 17 (depends on 15) The computer program product of claim 15 wherein the first machine learning model comprises a Contrastive Language-Image Pre-training (CLIP) model and the second machine learning model comprises a Visual Question Answering (VQA) model. ### Claim 18 A method comprising: obtaining an image of at least a portion of a computing device; processing the image of the portion of the computing device to generate two or more segments thereof for inclusion in a first data structure; determining a set of one or more labels to be provided as input to a first machine learning model, the one or more labels being associated with one or more designated defect types to be detected; applying the determined set of one or more labels and at least a portion of the first data structure to the first machine learning model to determine, for inclusion in a second data structure, probabilities of the one or more designated defect types being present in each of the two or more segments of the image of the portion of the computing device; determining, based at least in part on at least a portion of the second data structure, one or more input prompts for a second machine learning model; applying the determined one or more input prompts and at least a portion of the second data structure to the second machine learning model to generate answers to the determined one or more input prompts; and identifying, based at least in part on the generated answers to the determined one or more input prompts, whether the portion of the computing device includes any defects of the one or more designated defect types; wherein the method is performed by at least one processing device comprising a processor coupled to a memory. ### Claim 19 (depends on 18) The method of claim 18 wherein a segment size of the two or more segments of the image of the portion of the computing device is dynamically selected based at least in part on the one or more designated defect types to be detected. ### Claim 20 (depends on 18) The method of claim 18 wherein the first machine learning model comprises a Contrastive Language-Image Pre-training (CLIP) model and the second machine learning model comprises a Visual Question Answering (VQA) model.