# US 2022/0284476 A1 · Methods and apparatus for improving the selection of advertising

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0284476 A1
- Document ID: US20220284476A1
- Status: Published application
- Kind: A1
- Application number: 17/195,534
- Filed: 2021-03-08
- Published: 2022-09-08
- Earliest claimed: 2021-03-08
- Assignees: Walmart Apollo, LLC
- Applicants: Walmart Apollo, LLC
- Inventors: Peng Yang; Dong Xu; Yue Guo
- Web page: https://patentgrep.com/patent/US20220284476A1

## CPC

- G06Q 30/0257: {User requested}
- G06Q 30/0246: {Traffic}
- G06Q 30/0276: {Advertisement creation}
- G06Q 30/0277: {Online advertisement}
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR

## Abstract

The disclosed subject matter relates to a system and method for selecting/recommending ads based on a contextual bandit approach. The disclosed approach leverages various embedding vectors of item, search, page taxonomy trained based on traffic data via advanced deep learning models, and uses model signals (e.g. historical CTR, item price, rating, quality) from other ad placements. The learning mechanism on top of the current methodology to automatic chooses the best feature sets and adjust model performance over time. The contextual bandit model performs better with respect to CTR than the Thompson Sampling model, and achieves lower regret and faster convergence over time.

## Claims

### Claim 1

A system for retrieving marketing ads implementing a contextual bandit in response to a user query comprising:
a communication system;
a database containing a plurality of marketing ads; and,
a computing device operably connected to the database and the communication system, the computing device configured to:
assigning a first parameter set for each of the plurality of marketing ads;
receive an input query from a user;
determine at least one feature of the query;
filter the plurality of marketing ads from the database based on the at least one feature to create a first set of candidate marketing ads;
determine a context feature set parameter for each of the candidate marketing ads in the first set of candidate marketing ads;
select a second set of marketing ads based upon the context feature parameter of each of the candidate marketing ads in the first set;
transmit the second set of marketing ads to the user;
receive a reward metric; and,
update and optimize the first parameter set for each of the marketing ads in the second set as a function of the reward metric;
wherein the context feature set parameter is a function of at least the first parameter set.

### Claim 2 (depends on 1)

The method of claim 1, wherein the computing system is further configured to:
detect the query intent of the input query;
query the database based upon the query intent;
receive the query results from the database; and,
transmit the query results to the user along with second set of marketing ads in response to the input query.

### Claim 3 (depends on 1)

The system of claim 1, wherein the computing device is further configured to:
receive a second input query from the user;
determine at least one second feature of the second query;
filter the plurality of marketing ads from the database based on the at least one second feature to create a third set of candidate marketing ads;
determine an context feature set parameter for each of the candidate marketing ads in the third set of candidate marketing ads;
select a fourth set of marketing ads based upon the context feature set parameter of each of the candidate marketing ads in the third set;
transmit the fourth set of marketing ads to the user;
receive a second reward metric; and,
update and optimize the first parameter set for each of the marketing ads in the fourth set as a function of the reward.

### Claim 4 (depends on 1)

The system of claim 1, wherein the at least one feature is selected from the group consisting of taxonomy, title, taxonomy vector; product vector; and price sensitivity.

### Claim 5 (depends on 1)

The system of claim 1, wherein the reward metric is selected from the group consisting of views, clicks, add to cart and purchase.

### Claim 6 (depends on 1)

The system of claim 1, wherein the first parameter set are elements of a weight vector and wherein the contextual feature set parameter is a function of the weight vector.

### Claim 7 (depends on 1)

The system of claim 1, wherein the plurality of marketing ads each include at least one ad feature and the computing device is configured to filter the plurality of marketing ads based upon the at least one ad feature to create the first set of candidate marketing ads.

### Claim 8 (depends on 1)

The system of claim 1, wherein the computing device is further configured to apply a smoothing function to the updated contextual feature set parameter.

### Claim 9 (depends on 6)

The system of claim 6, wherein the computing device is further configured to create the weight vector as a function of a posterior distribution.

### Claim 10

A method for improving the selection of marketing ads, comprising:
assigning a first parameter set for each of the plurality of marketing ads;
receiving an input query from a user;
determining at least one feature of the query;
filtering the plurality of marketing ads from the database based on the at least one feature to create a first set of candidate marketing ads;
determining a context feature set parameter for each of the candidate marketing ads in the first set of candidate marketing ads;
selecting a second set of marketing ads based upon the context feature parameter of each of the candidate marketing ads in the first set;
transmitting the second set of marketing ads to the user;
receiving a reward metric; and,
updating and optimizing the first parameter set for each of the marketing ads in the second set as a function of the reward metric;
wherein the context feature set parameter is a function of at least the first parameter set.

### Claim 11 (depends on 10)

The method of claim 10, further comprising:
detecting the query intent of the input query;
querying the database based upon the query intent;
receiving the query results from the database; and,
transmitting the query results to the user along with second set of marketing ads in response to the input query.

### Claim 12 (depends on 10)

The method of claim 10, further comprising:
receiving a second input query from the user;
determining at least one second feature of the second query;
filtering the plurality of marketing ads from the database based on the at least one second feature to create a third set of candidate marketing ads;
determining an context feature set parameter for each of the candidate marketing ads in the third set of candidate marketing ads;
selecting a fourth set of marketing ads based upon the context feature set parameter of each of the candidate marketing ads in the third set;
transmitting the fourth set of marketing ads to the user;
receiving a second reward metric; and,
updating and optimizing the first parameter set for each of the marketing ads in the fourth set as a function of the reward.

### Claim 13 (depends on 10)

The method of claim 10, wherein the at least one feature is selected from the group consisting of taxonomy, title, taxonomy vector; product vector; and price sensitivity.

### Claim 14 (depends on 10)

The method of claim 10, wherein the reward metric is selected from the group consisting of views, clicks, add to cart and purchase.

### Claim 15 (depends on 10)

The method of claim 10, wherein the first parameter set are elements of a weight vector and wherein the contextual feature set parameter is a function of the weight vector.

### Claim 16 (depends on 10)

The method of claim 10, wherein the plurality of marketing ads each include at least one ad feature and the computing device is configured to filter the plurality of marketing ads based upon the at least one ad feature to create the first set of candidate marketing ads.

### Claim 17 (depends on 10)

The method of claim 10, further comprising the step of applying a smoothing function to the updated contextual feature set parameter.

### Claim 18

A non-transitory computer readable medium having instructions stored thereon, wherein the instructions, when executed by at least one processor, cause a device to perform operations comprising:
assigning a first parameter set for each of the plurality of marketing ads;
receiving an input query from a user;
determining at least one feature of the query;
filtering the plurality of marketing ads from the database based on the at least one feature to create a first set of candidate marketing ads;
determining a context feature set parameter for each of the candidate marketing ads in the first set of candidate marketing ads;
selecting a second set of marketing ads based upon the context feature parameter of each of the candidate marketing ads in the first set;
transmitting the second set of marketing ads to the user;
receiving a reward metric; and,
updating and optimizing the first parameter set for each of the marketing ads in the second set as a function of the reward metric;
wherein the context feature set parameter is a function of at least the first parameter set.

### Claim 19 (depends on 18)

The non-transitory computer readable medium of claim 18, wherein the instructions further perform the operations of:
detecting the query intent of the input query;
querying the database based upon the query intent;
receiving the query results from the database; and,
transmitting the query results to the user along with second set of marketing ads in response to the input query.

### Claim 20 (depends on 18)

The non-transitory computer readable medium of claim 18, wherein the instructions further perform the operations of:
receiving a second input query from the user;
determining at least one second feature of the second query;
filtering the plurality of marketing ads from the database based on the at least one second feature to create a third set of candidate marketing ads;
determining an context feature set parameter for each of the candidate marketing ads in the third set of candidate marketing ads;
selecting a fourth set of marketing ads based upon the context feature set parameter of each of the candidate marketing ads in the third set;
transmitting the fourth set of marketing ads to the user;
receiving a second reward metric; and,
updating and optimizing the first parameter set for each of the marketing ads in the fourth set as a function of the reward.
