# US 11,908,053 B2 · Method, non-transitory computer-readable storage medium, and apparatus for searching an image database > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,908,053 B2 - Document ID: US11908053B2 - Status: Granted - Kind: B2 - Application number: 17/333,707 - Filed: 2021-05-28 - Published: 2021-12-02 - Granted: 2024-02-20 - Earliest claimed: 2020-05-29 - Assignees: Camelot UK Bidco Limited - Applicants: Camelot UK Bidco Limited - Inventors: Sandra Mau; Joshua Song; Sabesan Sivapalan; Sreeja Krishnan - Web page: https://patentgrep.com/patent/US11908053B2 ## CPC - G06T 11/60: Creating or editing images; Combining images with text - G06F 16/583: using metadata automatically derived from the content - G06T 7/10: Segmentation; Edge detection (motion-based segmentation G06T7/215) - G06V 10/26: Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion - G06V 10/454: {Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]} - G06V 10/82: using neural networks - G06V 30/147: {Determination of region of interest} - G06T 2207/20084: Artificial neural networks [ANN] - G06T 2210/12: Bounding box - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A method for searching an image database, comprising receiving an adulterated image of an object, the adulterated image including object annotations for visual reference, applying a first neural network to the adulterated image, correlating a result of the applying the first neural network with each image of a reference database of images, the result including an edited image of the object and each image of the reference database of images including a reference object, and selecting, as a matching image, at least one image of the reference database of images having correlation values above a threshold correlation value. The method may include applying a masking to the adulterated image. The masking may include performing, via a second neural network, object recognition on the adulterated image, applying, based on the object recognition, computer vision to detect callout features relative to bounding boxes, and generating a contour mask of the object. ## Claims ### Claim 1 A method for searching an image database, comprising: receiving, by processing circuitry, an adulterated image of an object, the adulterated image including object annotations for visual reference; applying, by the processing circuitry, a first neural network to the adulterated image; correlating, by the processing circuitry, a result of the applying the first neural network with each image of a reference database of images, the result including an edited image of the object and each image of the reference database of images including a reference object; and selecting, by the processing circuitry and as a matching image, one or more images of the reference database of images having correlation values above a threshold correlation value, wherein the method further comprises applying, by the processing circuitry, a masking process to the adulterated image, including performing, by the processing circuitry and via a second neural network, object recognition on the adulterated image to recognize text-based descriptive features, applying, by the processing circuitry and based on the performing the object recognition, computer vision to detect callout features relative to bounding boxes containing the recognized text-based descriptive features, and generating, by the process circuitry and based on the bounding boxes and the detected callout features, a contour mask of the object. ### Claim 2 (depends on 1) The method according to claim 1, wherein the applying the first neural network includes editing, by the processing circuitry, the adulterated image to remove the object annotations, an output of the editing being the edited image of the object. ### Claim 3 (depends on 1) The method according to claim 1, further comprising editing, by the processing circuitry, the adulterated image based on the result of the applying the first neural network and a result of the applying the masking process, the result of the applying the masking process being the generated contour mask of the object. ### Claim 4 (depends on 1) The method according to claim 1, wherein the first neural network is a convolutional neural network configured to perform pixel-level classification. ### Claim 5 (depends on 1) The method according to claim 1, wherein the generating the contour mask of the object includes applying, by the processing circuitry, object segmentation to the adulterated image. ### Claim 6 A non-transitory computer-readable storage medium storing computer-readable instructions that, when executed by a computer, cause the computer to perform a method for searching an image database, comprising: receiving an adulterated image of an object, the adulterated image including object annotations for visual reference; applying a first neural network to the adulterated image; correlating a result of the applying the first neural network with each image of a reference database of images, the result including an edited image of the object and each image of the reference database of images including a reference object; and selecting, as a matching image, one or more images of the reference database of images having correlation values above a threshold correlation value, wherein the method further comprises applying a masking process to the adulterated image, including performing, via a second neural network, object recognition on the adulterated image to recognize text-based descriptive features, applying, based on the performing the object recognition, computer vision to detect callout features relative to bounding boxes containing the recognized text-based descriptive features, and generating, based on the bounding boxes and the detected callout features, a contour mask of the object. ### Claim 7 (depends on 6) The non-transitory computer-readable storage medium according to claim 6, wherein the applying the first neural network includes editing the adulterated image to remove the object annotations, an output of the editing being the edited image of the object. ### Claim 8 (depends on 6) The non-transitory computer-readable storage medium according to claim 6, further comprising editing the adulterated image based on the result of the applying the first neural network and a result of the applying the masking process, the result of the applying the masking process being the generated contour mask of the object. ### Claim 9 (depends on 6) The non-transitory computer-readable storage medium according to claim 6, wherein the first neural network is a convolutional neural network configured to perform pixel-level classification. ### Claim 10 (depends on 6) The non-transitory computer-readable storage medium according to claim 6, wherein the generating the contour mask of the object includes applying object segmentation to the adulterated image. ### Claim 11 An apparatus for performing a method for searching an image database, comprising: processing circuitry configured to receive an adulterated image of an object, the adulterated image including object annotations for visual reference, apply a first neural network to the adulterated image, correlate a result of the applying the first neural network with each image of a reference database of images, the result including an edited image of the object and each image of the reference database of images including a reference object, and select, as a matching image, one or more images of the reference database of images having correlation values above a threshold correlation value, wherein the processing circuitry is further configured to apply a masking process to the adulterated image by performing, via a second neural network, object recognition on the adulterated image to recognize text-based descriptive features, applying, based on the performing the object recognition, computer vision to detect callout features relative to bounding boxes containing the recognized text-based descriptive features, and generating, based on the bounding boxes and the detected callout features, a contour mask of the object. ### Claim 12 (depends on 11) The apparatus according to claim 11, wherein the processing circuitry is further configured to apply the first neural network by editing the adulterated image to remove the object annotations, an output of the editing being the edited image of the object. ### Claim 13 (depends on 11) The apparatus according to claim 11, wherein the processing circuitry is further configured to edit the adulterated image based on the result of the applying the first neural network and a result of the applying the masking process, the result of the applying the masking process being the generated contour mask of the object. ### Claim 14 (depends on 11) The apparatus according to claim 11, wherein the processing circuitry is further configured to generate the contour mask of the object by applying object segmentation to the adulterated image.