# US 2019/0171906 A1 · Digital Image Search Based On Arbitrary Image Features > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0171906 A1 - Document ID: US20190171906A1 - Status: Published application - Kind: A1 - Application number: 15/831,160 - Filed: 2017-12-04 - Published: 2019-06-06 - Earliest claimed: 2017-12-04 - Assignees: Adobe Inc. - Applicants: Adobe Inc. - Inventors: Shagun Sodhani; Nikaash Puri - Web page: https://patentgrep.com/patent/US20190171906A1 ## CPC - G06F 16/583: using metadata automatically derived from the content - G06F 16/5838: {using colour} - G06F 18/211: Selection of the most significant subset of features - G06F 18/2113: by ranking or filtering the set of features, e.g. using a measure of variance or of feature cross-correlation - G06F 18/22: Matching criteria, e.g. proximity measures - G06F 18/40: Software arrangements specially adapted for pattern recognition, e.g. user interfaces or toolboxes therefor - G06V 10/75: Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries - G06V 10/771: Feature selection, e.g. selecting representative features from a multi-dimensional feature space - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract In implementations of digital image search based on arbitrary image features, a server computing device maintains an images database of digital images, and includes an image search system that receives a search input as a digital image depicting image features, and receives search criteria of depicted image features in the digital image. The image search system can then determine similar images to the received digital image based on similarity criterion corresponding to the search criteria. A trained image model of the image search system is applied to determine an image feature representation of the received digital image. A feature mask model of the image search system is applied to the image feature representation to determine a masked feature representation of the received digital image. The masked feature representation of the received digital image is compared to a masked feature representation of each respective database image to identify the similar images. ## Claims ### Claim 1 In a digital medium environment for searching digital images based on arbitrary image features, a method implemented by at least one computing device of a service system, the method comprising: receiving a search input as a digital image depicting image features; receiving search criteria of one or more depicted image features in the digital image; determining similar images to the received digital image based on similarity criterion corresponding to the search criteria, the determining comprising: determining an image feature representation of the received digital image; determining a masked feature representation of the received digital image based on the image feature representation of the received digital image; and comparing the masked feature representation of the received digital image to a masked feature representation of each respective database digital image to identify the similar images to the received digital image based on the similarity criterion. ### Claim 2 (depends on 1) The method as recited in claim 1, further comprising: returning the similar images as one or more of the database digital images having similar representations of the masked feature representation for the received digital image and the masked feature representation of each respective database digital image. ### Claim 3 (depends on 2) The method as recited in claim 2, further comprising: identifying one or more image features depicted in a returned similar image as user feedback of the image features contributing to the identifying the similar images to the received digital image. ### Claim 4 (depends on 1) The method as recited in claim 1, further comprising: applying a trained image model of an image search system to determine the image feature representation as an image feature vector of the received digital image, the image feature vector representing the depicted image features in the received digital image; applying a feature mask model of the image search system to the image feature vector to determine the masked feature representation as a masked feature vector for the received digital image, the masked feature vector representing the similarity criterion of the depicted image features in the digital image; and the comparing comprising determining a Euclidian distance between the received digital image and each of the database digital images, the Euclidian distances determined based on a comparison of the masked feature vector for the received digital image to a masked feature vector of each respective database digital image, and the Euclidian distances identifying the similar images to the received digital image based on the similarity criterion. ### Claim 5 (depends on 4) The method as recited in claim 4, further comprising: returning the similar images as one or more of the database digital images having closest Euclidian distances between the masked feature vector for the received digital image and the masked feature vector of each respective database digital image. ### Claim 6 (depends on 1) The method as recited in claim 1, wherein: each of the similarity criterion is represented by a similarity criterion mask; and a combination of two or more similarity criterion masks represent the similarity criterion corresponding to two or more of the search criteria for the depicted image features in the received digital image. ### Claim 7 (depends on 1) The method as recited in claim 1, further comprising: composing together one or more similarity criterion masks that each correspond to the search criteria, the composed one or more similarity criterion masks usable for the determining the similar images based on a combination of the similarity criterion. ### Claim 8 (depends on 1) The method as recited in claim 1, further comprising: determining that the search criteria does not have a similarity criterion mask corresponding to the similarity criterion in the image search system; and determining the similarity criterion mask corresponding to the search criteria. ### Claim 9 (depends on 1) The method as recited in claim 1, further comprising: determining suggested search criteria for the received digital image based on the image feature representation that represents the depicted image features in the received digital image; and providing the suggested search criteria to a user for selection as the search criteria and basis for the determining the similar images. ### Claim 10 A server computing device implemented to search digital images based on arbitrary image features, the server computing device comprising: a memory to maintain an images database of the digital images; a processor system configured to execute an image search system implemented to perform operations comprising to: receive a search input as a digital image depicting image features; receive search criteria of one or more depicted image features in the digital image; determine similar images to the received digital image based on similarity criterion corresponding to the search criteria, the operation to determine comprising to: apply a trained image model of the image search system to determine an image feature representation of the received digital image; apply a feature mask model of the image search system to the image feature representation to determine a masked feature representation of the received digital image; and compare the masked feature representation of the received digital image to a masked feature representation of each respective database digital image to identify the similar images to the received digital image based on the similarity criterion. ### Claim 11 (depends on 10) The server computing device as recited in claim 10, wherein: the image feature representation is an image feature vector generated by the trained image model and represents the depicted image features in the received digital image; and the masked feature representation is a masked feature vector generated by the feature mask model and represents the similarity criterion of the depicted image features in the digital image. ### Claim 12 (depends on 11) The server computing device as recited in claim 11, wherein: the operation to determine the similar images comprise to determine a Euclidian distance between the received digital image and each of the database digital images, the Euclidian distances identifying the similar images to the received digital image based on the similarity criterion; and the Euclidian distances are determined based on a comparison of the masked feature vector for the received digital image to a masked feature vector of each respective database digital image. ### Claim 13 (depends on 12) The server computing device as recited in claim 12, wherein the image search system is implemented to return the similar images as one or more of the database digital images having closest Euclidian distances between the masked feature vector for the received digital image and the masked feature vector of each respective database digital image. ### Claim 14 (depends on 13) The server computing device as recited in claim 13, wherein the image search system is implemented to identify one or more image features depicted in a returned similar image as user feedback of the image features contributing to the determination of the similar image to the received digital image. ### Claim 15 (depends on 10) The server computing device as recited in claim 10, wherein: each of the similarity criterion is represented by a similarity criterion mask; and a combination of two or more similarity criterion masks represent the similarity criterion corresponding to two or more of the search criteria for the depicted image features in the received digital image. ### Claim 16 (depends on 10) The server computing device as recited in claim 10, wherein the image search system is implemented to compose together one or more similarity criterion masks that each correspond to the search criteria, the composed one or more similarity criterion masks usable to determine the similar images based on a combination of the similarity criterion. ### Claim 17 (depends on 10) The server computing device as recited in claim 10, wherein the image search system is implemented to: determine that the search criteria does not have a similarity criterion mask corresponding to the similarity criterion in the image search system; and apply the feature mask model of the image search system to determine the similarity criterion mask corresponding to the search criteria. ### Claim 18 (depends on 10) The server computing device as recited in claim 10, wherein the image search system is implemented to: determine suggested search criteria for the received digital image based on the image feature representation that represents the depicted image features in the received digital image; and provide the suggested search criteria to a user for selection as the search criteria and basis for the determination of the similar images. ### Claim 19 A system for searching digital images based on arbitrary image features, the system comprising: means for storing the digital images in an images database; means for determining similar images to a received digital image based on similarity criterion corresponding to the search criteria, the means for determining comprising: means for determining an image feature representation of the received digital image; means for determining a masked feature representation for the received digital image based on the image feature representation of the received digital image; and means for comparing the masked feature representation of the received digital image to a masked feature representation of each respective database digital image to identify the similar images to the received digital image based on the similarity criterion. ### Claim 20 (depends on 19) The system as recited in claim 19, further comprising: means for returning the similar images as one or more of the database digital images having similar representations of the masked feature representation for the received digital image and the masked feature representation of each respective database digital image.