# US 11,907,987 B2 · Determining visually similar products > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,907,987 B2 - Document ID: US11907987B2 - Status: Granted - Kind: B2 - Application number: 17/550,245 - Filed: 2021-12-14 - Published: 2022-08-11 - Granted: 2024-02-20 - Earliest claimed: 2020-01-22 - Assignees: Home Depot Product Authority, LLC - Applicants: Home Depot Product Authority, LLC - Inventors: Estelle Afshar; Matthew Hagen; Huiming Qu - Web page: https://patentgrep.com/patent/US11907987B2 ## CPC - G06Q 30/0623: {by investigating goods or services} - G06F 18/22: Matching criteria, e.g. proximity measures - G06F 18/253: {of extracted features} - G06N 3/045: Combinations of networks - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/08: Learning methods - G06N 3/09: Supervised learning - G06N 3/0985: Hyperparameter optimisation; Meta-learning; Learning-to-learn - G06N 20/10: using kernel methods, e.g. support vector machines [SVM] - G06V 10/761: {Proximity, similarity or dissimilarity measures} - G06V 10/806: {of extracted features} - G06Q 30/0631: {Recommending goods or services} - G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A computer-implemented method for determining image similarity includes determining, by a first neural network, a first feature value associated with a first characteristic of a first product based on an image of the first product. The method also includes determining, by a second neural network, a second feature value associated with a second characteristic of the first product based on the image of the first product. The method further involves calculating a first vector space distance between the first feature value and a third feature value associated with the first characteristic of a second product, and calculating a second vector space distance between the second feature value and a fourth feature value associated with the second characteristic of the second product. Additionally, the method includes determining a similarity value based on the first vector space distance and the second vector space distance. ## Claims ### Claim 1 A computer-implemented method for determining image similarity, the method comprising: determining, by a first neural network, a first feature value associated with a first characteristic of a first image; determining, by a second neural network, a second feature value associated with a second characteristic of the first image; calculating a first distance between the first feature value and a third feature value associated with the first characteristic of a second image; calculating a second distance between the second feature value and a fourth feature value associated with the second characteristic of the second image; and based on a comparison of the first distance to the second distance, displaying, on a user interface, a representation of the first image in association with the second image. ### Claim 2 (depends on 1) The method of claim 1, wherein the representation of the first image is displayed in a first location within a list of images, the method further comprising: receiving a feature weighting value associated with the first characteristic; determining a first weighted distance based at least in part on the first distance and the feature weighting value; calculating a weighted similarity value based on the first weighted distance and the second distance; and determining a second location within the list of images at which to display the first image, based at least in part on the weighted similarity value. ### Claim 3 (depends on 1) The method of claim 1, wherein the first neural network is independent from the second neural network. ### Claim 4 (depends on 1) The method of claim 1, wherein the first image comprises a first product, and the second image comprises a second product. ### Claim 5 (depends on 1) The method of claim 1, wherein the second characteristic is substantially independent from the first characteristic. ### Claim 6 (depends on 1) The method of claim 1, wherein calculating the first distance comprises at least one of: calculating, as the first distance, a cosine similarity value between the first feature value and the third feature value; calculating, as the first distance, a Euclidean distance between the first feature value and the third feature value; or calculating, as the first distance, a Chebyshev distance between the first feature value and the third feature value. ### Claim 7 (depends on 1) The method of claim 1, wherein the first neural network is configured to extract one or more feature values representative of the first characteristic from input images. ### Claim 8 (depends on 1) The method of claim 1, wherein the second neural network is configured to extract one or more feature values representative of the second characteristic from input images. ### Claim 9 (depends on 1) The method of claim 1, further comprising: determining, by a third neural network, a fifth feature value associated with a third characteristic of the first image. ### Claim 10 (depends on 1) The method of claim 1, wherein the first characteristic includes color information determinable based on an input image. ### Claim 11 (depends on 1) The method of claim 1, wherein the first characteristic includes shape information determinable based on an input image. ### Claim 12 (depends on 1) The method of claim 1, wherein the first characteristic includes pattern information determinable based on an input image. ### Claim 13 (depends on 1) The method of claim 1, wherein the first characteristic includes style information determinable based on an input image. ### Claim 14 (depends on 1) The method of claim 1, wherein the comparison comprises determination of a first similarity value, the method further comprising: determining, by the first neural network, a fifth feature value associated with the first characteristic of a third image; determining, by the second neural network, a sixth feature value associated with the second characteristic of the third image; determining a third distance between the third feature value associated with the second image and the fifth feature value associated with the third image; determining a fourth distance between the fourth feature value associated with the second image and the sixth feature value associated with the third image; determining a second similarity value based on the third distance and the fourth distance; and based on the second similarity value being greater than the first similarity value, displaying, on the user interface, a representation of images that are visually similar to the second image in descending order, in which the third image precedes the first image. ### Claim 15 A computer-implemented method for determining image similarity, the method comprising: retrieving a set of weighting values based on a product category for a first product; determining, by a first feature extractor, a first feature value associated with a first characteristic of an image of the first product; determining, by a second feature extractor, a second feature value associated with a second characteristic of the first product image; determining a first weighted distance based on the first feature value, a third feature value associated with the first characteristic of a second product image, and a first weighting value of the set of weighting values; determining a second weighted distance based on the second feature value, a fourth feature value associated with the second characteristic of the second product image, and a second weighting value of the set of weighting values; and in response to a comparison of the first weighted distance and the second weighted distance, displaying, on a user interface, a representation of the second product image in association with the first product image. ### Claim 16 (depends on 15) The method of claim 15, wherein the comparison comprises: determining a similarity value based on the first weighted distance and the second weighted distance; and comparing the similarity value to a threshold value. ### Claim 17 (depends on 15) The method of claim 15, wherein the representation of the first product image is displayed in a first location within a list of products, the method further comprising: receiving a user-specified feature weighting value associated with the first characteristic based on user input; determining a third weighted distance based on the first feature value, the user-specified feature weighting value, and the third feature value; and based on a comparison of the third weighted distance and the second weighted distance, determining a second location within the list of products at which to display the first product image. ### Claim 18 (depends on 15) The method of claim 15, wherein determining the first weighted distance comprises adjusting a first distance based on the first weighting value, the first distance calculated as at least one of: a cosine similarity value between the first feature value and the third feature value; a Euclidean distance between the first feature value and the third feature value; or a Chebyshev distance between the first feature value and the third feature value. ### Claim 19 (depends on 15) The method of claim 15, wherein the first feature extractor is a convolutional neural network.