# US 2022/0020065 A1 · Ad Exchange Bid Optimization with Reinforcement Learning

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0020065 A1
- Document ID: US20220020065A1
- Status: Published application
- Kind: A1
- Application number: 17/379,959
- Filed: 2021-07-19
- Published: 2022-01-20
- Earliest claimed: 2020-07-19
- Assignees: Zeta Global Corp.
- Applicants: Zeta Global Corp.
- Inventors: Danny Portman; Zachary D. Jones; David Rose
- Web page: https://patentgrep.com/patent/US20220020065A1

## CPC

- G06Q 30/0275: {Auctions}
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/092: Reinforcement learning
- G06N 3/0985: Hyperparameter optimisation; Meta-learning; Learning-to-learn
- G06Q 30/0246: {Traffic}
- G06Q 30/0205: {based on location or geographical consideration}
- G06Q 30/0249: {based upon budgets or funds}
- G06Q 30/0256: {User search}
- G06Q 30/0276: {Advertisement creation}
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A system for training a bidding model comprising: a plurality of tactics stored on at least one database; a plurality of hyperparameters; in response to an available inventory from a publisher relayed through a real time bid server, computing a bid on the available inventory; sending the bid to the real time bid server; receiving an auction result in response to the bid; calculating a plurality of rewards based on the auction result and the tactics; calculate a plurality of q values based on the rewards; calculate a plurality of losses; backpropogating the losses through the bidding model.

## Claims

### Claim 1

A system for training a bidding model comprising:
a plurality of tactics stored on at least one database;
a plurality of hyperparameters;
in response to an available inventory from a publisher relayed through a real time bid server, computing a bid on the available inventory;
sending the bid to the real time bid server;
receiving an auction result in response to the bid;
calculating a plurality of rewards based on the auction result and the tactics;
calculate a plurality of q values based on the rewards;
calculate a plurality of losses;
backpropogating the losses through the bidding model.
