# US 2023/0057167 A1 · Motif-based image classification

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0057167 A1
- Document ID: US20230057167A1
- Status: Published application
- Kind: A1
- Application number: 17/889,752
- Filed: 2022-08-17
- Published: 2023-02-23
- Earliest claimed: 2021-08-18
- Assignees: Home Depot Product Authority, LLC
- Applicants: Home Depot Product Authority, LLC
- Inventors: Aron YU; Estelle AFSHAR
- Web page: https://patentgrep.com/patent/US20230057167A1

## CPC

- G06V 10/761: {Proximity, similarity or dissimilarity measures}
- G06V 10/772: Determining representative reference patterns, e.g. averaging or distorting patterns; Generating dictionaries
- G06V 10/774: Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
- G06V 10/778: Active pattern-learning, e.g. online learning of image or video features
- G06V 10/945: {User interactive design; Environments; Toolboxes}
- G06V 20/70: Labelling scene content, e.g. deriving syntactic or semantic representations
- G06V 30/413: Classification of content, e.g. text, photographs or tables
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A method for displaying images similar to a selected image includes receiving, from a user, a selection of an anchor image, generating, using a machine learning model, an anchor embeddings set for the anchor image and respective candidate embeddings sets for a plurality of candidate images. The method also includes calculating a distance between the anchor embeddings set and each of the plurality of candidate embeddings sets and displaying at least one of the plurality of candidate images based on the calculated distance.

## Claims

### Claim 1

A method comprising:
receiving, by a server system from a user, a selection of an anchor image;
generating, by the server system using a machine learning model, an anchor embeddings set for the anchor image, the anchor embeddings set indicative of binary values associated with the anchor image for a set of visual attributes;
generating, by the server system, using the machine learning model, respective candidate embeddings sets for a plurality of candidate images;
calculating a distance between the anchor embeddings set and each of the plurality of candidate embeddings sets; and
displaying at least one of the plurality of candidate images based on the calculated distance.

### Claim 2 (depends on 1)

The method of claim 1, wherein receiving the selection of the anchor image comprises:
receiving, by the server system from the user, a selection of an anchor document; and
deriving the anchor image from the anchor document by:
retrieving a set of images associated with the anchor document;
determining an alignment of each image in the set of images;
receiving an indication of a standard alignment; and
selecting, as the anchor image, the image in the set of images having the standard alignment.

### Claim 3 (depends on 2)

The method of claim 2, wherein the indication of the standard alignment is based on a document type of the anchor document.

### Claim 4 (depends on 2)

The method of claim 2, wherein deriving the anchor image further comprises:
determining that two or more images of the set of images have the standard alignment;
determining a confidence score associated with the alignment determination for each of the two or more images; and
selecting the anchor image as the image of the two or more images with a highest confidence score.

### Claim 5 (depends on 1)

The method of claim 1, wherein the displayed at least one candidate image corresponds to the candidate embeddings set that is closest to the anchor embeddings set.

### Claim 6 (depends on 1)

The method of claim 1, wherein each candidate embeddings set is indicative of binary values associated with each candidate image for the set of visual attributes.

### Claim 7 (depends on 6)

The method of claim 6, wherein the binary values represent a motif for each candidate image, the motif comprising a combination of visual attributes.

### Claim 8

A method comprising:
training a machine learning model to receive an input image and to output binary values for a plurality of visual attributes of the input image;
receiving, by a server system from a user, a selection of an anchor image;
generating, by the server system using the machine learning model, an anchor embeddings set for the anchor image;
generating, by the machine learning model, respective candidate embeddings sets for a plurality of candidate images, each candidate embeddings set associated with a candidate image;
calculating a distance between the anchor embeddings set and each of the plurality of candidate embeddings sets; and
displaying at least one of the plurality of candidate images based on the calculated distance.

### Claim 9 (depends on 8)

The method of claim 8, wherein the machine learning model generates embeddings for an image with a first layer of the model that, during training, is input to a second layer that outputs the binary values for the plurality of visual attributes of the image.

### Claim 10 (depends on 8)

The method of claim 8, wherein training the machine learning model comprises minimizing a cross-entropy loss function, wherein a loss of the loss function comprises a sum of losses respective of the binary values for the plurality of visual attributes.

### Claim 11 (depends on 8)

The method of claim 8, wherein the plurality of visual attributes are generated by:
defining a set of images, each image associated with a document;
deriving, from each image of the set of images, one or more pre-determined labels;
standardizing the one or more pre-determined labels;
generating label classes based on the standardized labels; and
flattening the label classes with the pre-determined labels to generate the set of visual attributes.

### Claim 12 (depends on 11)

The method of claim 11, wherein flattening comprises consolidating the one or more standardized labels and the label classes into the set of visual attributes,
wherein each visual attribute comprises at least one label class and at least one standardized label, and
wherein the binary value for each visual attribute is indicative of an image associated with the binary value having the visual attribute.

### Claim 13 (depends on 8)

The method of claim 8, wherein receiving the selection of the anchor image comprises:
receiving, by the server system from the user, a selection of an anchor document; and
deriving the anchor image from the anchor document by:
retrieving a set of images associated with the anchor document;
determining an alignment of each image in the set of images;
receiving an indication of a standard alignment; and
selecting, as the anchor image, the image in the set of images having the standard alignment.

### Claim 14 (depends on 8)

The method of claim 8, wherein the displayed at least one candidate image corresponds to the candidate embeddings set closest to the anchor embeddings set.

### Claim 15 (depends on 8)

The method of claim 8, wherein displaying the at least one candidate image comprises displaying a candidate document associated with each of the at least one candidate image.

### Claim 16

A method comprising:
deriving a set of images from a set of documents;
deriving, from each image of the set of images, one or more pre-determined labels;
standardizing the one or more pre-determined labels;
generating label classes based on the standardized labels; and
flattening the label classes with the standardized labels to generate a defined set comprising a plurality of visual attributes;
training a machine learning model to receive an input image and to output binary values for the plurality of visual attributes of the input image;
receiving, by a server system from a user, a selection of an anchor image;
generating, by the server system using a machine learning model, an anchor embeddings set for the anchor image, the anchor embeddings set indicative of binary values associated with the anchor image for the set of visual attributes;
generating, by server system using the machine learning model, respective candidate embeddings sets for a plurality of candidate images, each candidate embeddings set indicative of binary values associated with each candidate image for the set of visual attributes;
calculating a distance between the anchor embeddings set and each of the plurality of candidate embeddings sets; and
displaying at least one of the plurality of candidate images based on the calculated distance.

### Claim 17 (depends on 16)

The method of claim 16, wherein flattening comprises consolidating the one or more standardized labels and the label classes into the set of visual attributes,
wherein each visual attribute comprises at least one label class and at least one standardized label, and
wherein the binary value for each visual attribute is indicative of an image associated with the binary value having the visual attribute.

### Claim 18 (depends on 16)

The method of claim 16, wherein receiving the selection of the anchor image comprises:
receiving, by the server system from the user, a selection of an anchor document; and
deriving the anchor image from the anchor document by:
retrieving a set of images associated with the anchor document;
determining an alignment of each image in the set of images;
receiving an indication of a standard alignment; and
selecting, as the anchor image, the image in the set of images having the standard alignment.

### Claim 19 (depends on 18)

The method of claim 18, wherein deriving the anchor image further comprises:
determining that two or more images of the set of images have the standard alignment;
determining a confidence score associated with the alignment determination for each of the two or more images; and
selecting the anchor image as the image of the two or more images with a highest confidence score.

### Claim 20 (depends on 16)

The method of claim 16, wherein displaying the at least one candidate image comprises displaying a candidate document associated with each of the at least one candidate image.
