# US 2021/0240739 A1 · Automatically determining items to include in a variant group

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0240739 A1
- Document ID: US20210240739A1
- Status: Published application
- Kind: A1
- Application number: 16/779,473
- Filed: 2020-01-31
- Published: 2021-08-05
- Earliest claimed: 2020-01-31
- Assignees: Walmart Apollo, LLC
- Applicants: Walmart Apollo, LLC
- Inventors: Yanxin Pan; Swagata Chakraborty; Abhinandan Krishnan; Abon Chaudhuri; Aakash Mayur Mehta; Edison Mingtao Zhang; Kyu Bin Kim
- Web page: https://patentgrep.com/patent/US20210240739A1

## CPC

- G06F 16/285: {Clustering or classification}
- G06F 16/24553: {of query operations}
- G06F 16/55: Clustering; Classification
- G06F 16/5846: {using extracted text}
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G06N 20/00: Machine learning
- G06Q 30/0603: {Catalogue creation or management}
- G06Q 30/0641: {utilising user interfaces specially adapted for shopping}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR

## Abstract

A method including obtaining image data and attribute information of a first item in an item catalog. The method also can include generating candidate variant items from the item catalog for the first item using a combination of (a) a k-nearest neighbors approach to search for first candidate variant items based on text embeddings for the attribute information of the first item, and (b) an elastic search approach to search for second candidate variant items based on image embeddings for the image data of the first item. The method additionally can include performing respective classifications based on respective pairs comprising the first item and each of the candidate variant items to filter the candidate variant items. The method further can include determining a respective distance between the first item and each of the candidate variant items, as filtered. The method additionally can include determining one or more items in the candidate variant items, as filtered, to include in a variant group for the first item, based on a decision function using a predetermined threshold and the respective distance for the each of the candidate variant items, as filtered. Other embodiments are described.

## Claims

### Claim 1

A system comprising:
one or more processors; and
one or more non-transitory computer-readable media storing computing instructions configured to run on the one or more processors and perform:
obtaining image data and attribute information of a first item in an item catalog;
generating candidate variant items from the item catalog for the first item using a combination of (a) a k-nearest neighbors approach to search for first candidate variant items based on text embeddings for the attribute information of the first item, and (b) an elastic search approach to search for second candidate variant items based on image embeddings for the image data of the first item;
performing respective classifications based on respective pairs comprising the first item and each of the candidate variant items to filter the candidate variant items;
determining a respective distance between the first item and each of the candidate variant items, as filtered; and
determining one or more items in the candidate variant items, as filtered, to include in a variant group for the first item, based on a decision function using a predetermined threshold and the respective distance for the each of the candidate variant items, as filtered.

### Claim 2 (depends on 1)

The system of claim 1, wherein the computing instructions are further configured to perform, before generating the candidate variant items, performing a pre-processing comprising:
generating the text embeddings from the attribute information of the first item; and
generating the image embeddings from the image data of the first item.

### Claim 3 (depends on 2)

The system of claim 2, wherein:
the text embeddings are generated using an LSTM model.

### Claim 4 (depends on 2)

The system of claim 2, wherein:
the image embeddings are generated using a VGG16 model.

### Claim 5 (depends on 1)

The system of claim 1, wherein:
the k-nearest neighbors approach comprises using a non-metric space library (NMSLIB).

### Claim 6 (depends on 1)

The system of claim 1, wherein:
the combination comprises a union of the first candidate variant items generated using the k-nearest neighbors approach and the second candidate variant items generated using the elastic search approach.

### Claim 7 (depends on 1)

The system of claim 1, wherein performing the respective classifications further comprises:
performing the respective classifications using a logistic regression model with L1 regularization based on respective product type classification embeddings for the candidate variant items.

### Claim 8 (depends on 1)

The system of claim 1, wherein determining a respective distance further comprises:
determining the respective distance the respective distance using a Siamese network.

### Claim 9 (depends on 1)

The system of claim 1, wherein the computing instructions are further configured to perform:
determining a coherence score for the variant group to select one or more supergroups from among multiple variant groups, wherein the multiple variant groups comprise the variant group.

### Claim 10 (depends on 1)

The system of claim 1, wherein the computing instructions are further configured to perform:
determining a Jaccard similarity score between multiple variant groups, wherein the multiple variant groups comprise the variant group.

### Claim 11

A method being implemented via execution of computing instructions configured to run at one or more processors and stored at one or more non-transitory computer-readable media, the method comprising:
obtaining image data and attribute information of a first item in an item catalog;
generating candidate variant items from the item catalog for the first item using a combination of (a) a k-nearest neighbors approach to search for first candidate variant items based on text embeddings for the attribute information of the first item, and (b) an elastic search approach to search for second candidate variant items based on image embeddings for the image data of the first item;
performing respective classifications based on respective pairs comprising the first item and each of the candidate variant items to filter the candidate variant items;
determining a respective distance between the first item and each of the candidate variant items, as filtered; and
determining one or more items in the candidate variant items, as filtered, to include in a variant group for the first item, based on a decision function using a predetermined threshold and the respective distance for the each of the candidate variant items, as filtered.

### Claim 12 (depends on 11)

The method of claim 11, further comprising, before generating the candidate variant items, performing a pre-processing comprising:
generating the text embeddings from the attribute information of the first item; and
generating the image embeddings from the image data of the first item.

### Claim 13 (depends on 12)

The method of claim 12, wherein:
the text embeddings are generated using an LSTM model.

### Claim 14 (depends on 12)

The method of claim 12, wherein:
the image embeddings are generated using a VGG16 model.

### Claim 15 (depends on 11)

The method of claim 11, wherein:
the k-nearest neighbors approach comprises using a non-metric space library (NMSLIB).

### Claim 16 (depends on 11)

The method of claim 11, wherein:
the combination comprises a union of the first candidate variant items generated using the k-nearest neighbors approach and the second candidate variant items generated using the elastic search approach.

### Claim 17 (depends on 11)

The method of claim 11, wherein performing the respective classifications further comprises:
performing the respective classifications using a logistic regression model with L1 regularization based on respective product type classification embeddings for the candidate variant items.

### Claim 18 (depends on 11)

The method of claim 11, wherein determining a respective distance further comprises:
determining the respective distance the respective distance using a Siamese network.

### Claim 19 (depends on 11)

The method of claim 11, further comprising:
determining a coherence score for the variant group to select one or more supergroups from among multiple variant groups, wherein the multiple variant groups comprise the variant group.

### Claim 20 (depends on 11)

The method of claim 11, further comprising:
determining a Jaccard similarity score between multiple variant groups, wherein the multiple variant groups comprise the variant group.