# US 12,008,054 B2 · Systems and methods for determining and utilizing search token importance using machine learning architectures

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,008,054 B2
- Document ID: US12008054B2
- Status: Granted
- Kind: B2
- Application number: 17/588,663
- Filed: 2022-01-31
- Published: 2023-09-07
- Granted: 2024-06-11
- Earliest claimed: 2022-01-31
- Assignees: Walmart Apollo, LLC
- Applicants: Walmart Apollo, LLC
- Inventors: Satya Chembolu; Tony Eilo Lee; Ciya Liao; Kaushal Yagnik
- Web page: https://patentgrep.com/patent/US12008054B2

## CPC

- G06F 16/953: Querying, e.g. by the use of web search engines
- G06N 3/08: Learning methods
- G06F 16/3334: {Selection or weighting of terms from queries, including natural language queries}
- G06F 16/9032: Query formulation
- G06F 16/9035: Filtering based on additional data, e.g. user or group profiles
- G06F 16/9532: Query formulation
- G06F 16/9535: Search customisation based on user profiles and personalisation
- G06N 5/022: Knowledge engineering; Knowledge acquisition
- G06F 40/216: using statistical methods
- G06F 40/284: Lexical analysis, e.g. tokenisation or collocates
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/048: Activation functions
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Systems and methods including one or more processors and one or more non-transitory storage devices storing computing instructions configured to run on the one or more processors and execute functions comprising: receiving, at a search engine, a search query comprising tokens; determining, using a weight determination model, token weights for the tokens included in the search query; comparing the token weights associated with the tokens to a probability threshold; identifying the tokens that satisfy the probability threshold; and utilizing the tokens that satisfy the probability threshold to identify the search results. Other embodiments are disclosed herein.

## Claims

### Claim 1

A system comprising:
one or more processors; and
one or more non-transitory computer-readable media storing computing instructions, that when executed on the one or more processors, cause the one or more processors to execute operations comprising:
receiving, at a search engine, a search query comprising tokens;
determining, using a weight determination model of a machine learning architecture, token weights for the tokens included in the search query, wherein each token weight predicts a probability that a corresponding token will match metadata of a positively engaged item included in search results that are presented in response to the search query, wherein the weight determination model is configured to identify extraneous tokens based on at least the token weights or the metadata, wherein the metadata comprises at least one of names, titles, or keywords associated with items of a catalog, and wherein the positively engaged item comprises an item in a set of items that was at least one of selected, viewed, ordered, purchased, or added to a cart by a user who submitted a previous search query;
comparing the token weights associated with the tokens to a probability threshold;
segregating the extraneous tokens based on at least the token weights or the metadata;
identifying the tokens that satisfy the probability threshold; and
utilizing the tokens that satisfy the probability threshold to identify the search results.

### Claim 2 (depends on 1)

The system of claim 1, wherein the tokens that do not satisfy the probability threshold comprise negatively engaged items that are not utilized to identify the search results.

### Claim 3 (depends on 1)

The system of claim 1,
wherein:
the weight determination model is trained using historical search data to generate the token weights for the tokens; and
the weight determination model eliminates the extraneous tokens in the search query based on the token weights.

### Claim 4 (depends on 1)

The system of claim 1, wherein the weight determination model comprises:
(a) a language processing model configured to:
receive the tokens included with the search query; and
generate token vectors based on the tokens;
(b) a sequence model that is configured to:
receive the token vectors from the language processing model; and
generate token weight vectors corresponding to the token weights for the tokens;
and
(c) a classification layer that is configured to:
receive the token weight vectors from the sequence model; and
map the token weight vectors to probability values representing the token weights.

### Claim 5 (depends on 1)

The system of claim 1, wherein:
the search engine is configured to identify a recall set of search results; and
the tokens that satisfy the probability threshold are utilized to identify the recall set of search results.

### Claim 6 (depends on 1)

The system of claim 1, wherein the computing instructions, when executed on the one or more processors, further cause the one or more processors to execute operations comprising:
determining a number of the tokens that satisfy the probability threshold; and
comparing the number of the tokens that satisfy the probability threshold to a maximum token threshold, wherein the maximum token threshold indicates a maximum number of the tokens that can be utilized to identify the search results.

### Claim 7 (depends on 6)

The system of claim 6, wherein:
in response to determining that the number of the tokens that satisfy the probability threshold exceeds the maximum token threshold, selecting a subset of the tokens that satisfy the probability threshold to identify the search results; and
the subset of the tokens is selected based on the token weights.

### Claim 8 (depends on 1)

The system of claim 1, wherein:
the weight determination model comprises a sequence model;
the sequence model is trained using training samples that are derived from historical search data; and
the training samples comprise previous search queries submitted for positively engaged items.

### Claim 9 (depends on 8)

The system of claim 8, wherein:
each of the previous search queries comprises one or more tokens;
each of the one or more tokens are assigned a label; and
the label indicates whether the corresponding token matched the metadata of a corresponding positively engaged item.

### Claim 10 (depends on 8)

The system of claim 8, wherein:
the training samples enable the sequence model to assign the token weights to the tokens based on a context in which the tokens are used in a given search query, such that an identical token used in both a first search query and a second search query is assigned different token weights based on the context.

### Claim 11

A method implemented via execution of computing instructions configured to run at one or more processors and stored on non-transitory computer-readable media, the method comprising:
receiving, at a search engine, a search query comprising tokens;
determining, using a weight determination model of a machine learning architecture, token weights for the tokens included in the search query, wherein each token weight predicts a probability that a corresponding token will match metadata of a positively engaged item included in search results that are presented in response to the search query, wherein the weight determination model is configured to identify extraneous tokens based on at least the token weights or the metadata, wherein the metadata comprises at least one of names, titles, or keywords associated with items of a catalog, and wherein the positively engaged item comprises an item in a set of items that was at least one of selected, viewed, ordered, purchased, or added to a cart by a user who submitted a previous search query;
comparing the token weights associated with the tokens to a probability threshold;
segregating the extraneous tokens based on at least the token weights or the metadata;
identifying the tokens that satisfy the probability threshold; and
utilizing the tokens that satisfy the probability threshold to identify the search results.

### Claim 12 (depends on 11)

The method of claim 11, wherein the tokens that do not satisfy the probability threshold comprise negatively engaged items that are not utilized to identify the search results.

### Claim 13 (depends on 1)

The method of claim 1,
wherein:
the weight determination model is trained using historical search data to generate the token weights for the tokens; and
the weight determination model eliminates the extraneous tokens in the search query based on the token weights.

### Claim 14 (depends on 11)

The method of claim 11, wherein the weight determination model comprises:
(a) a language processing model configured to:
receive the tokens included with the search query; and
generate token vectors based on the tokens;
(b) a sequence model that is configured to:
receive the token vectors from the language processing model; and
generate token weight vectors corresponding to the token weights for the tokens;
and
(c) a classification layer that is configured to:
receive the token weight vectors from the sequence model; and
map the token weight vectors to probability values representing the token weights.

### Claim 15 (depends on 11)

The method of claim 11, wherein:
the search engine is configured to identify a recall set of search results; and
the tokens that satisfy the probability threshold are utilized to identify the recall set of search results.

### Claim 16 (depends on 11)

The method of claim 11 further comprising:
determining a number of the tokens that satisfy the probability threshold; and
comparing the number of the tokens that satisfy the probability threshold to a maximum token threshold, wherein the maximum token threshold indicates a maximum number of the tokens that can be utilized to identify the search results.

### Claim 17 (depends on 16)

The method of claim 16, wherein:
in response to determining that the number of the tokens that satisfy the probability threshold exceeds the maximum token threshold, selecting a subset of the tokens that satisfy the probability threshold to identify the search results; and
the subset of the tokens is selected based on the token weights.

### Claim 18 (depends on 11)

The method of claim 11, wherein:
the weight determination model comprises a sequence model;
the sequence model is trained using training samples that are derived from historical search data; and
the training samples comprise previous search queries submitted for positively engaged items.

### Claim 19 (depends on 18)

The method of claim 18, wherein:
each of the previous search queries comprises one or more tokens;
each of the one or more tokens are assigned a label; and
the label indicates whether the corresponding token matched the metadata of a corresponding positively engaged item.

### Claim 20 (depends on 18)

The method of claim 18, wherein:
the training samples enable the sequence model to assign the token weights to the tokens based on a context in which the tokens are used in a given search query, such that an identical token used in both a first search query and a second search query is assigned different token weights based on the context.