Systems and methods for targeting bid and position for a keyword

A system for targeting bid and position for a keyword by tracking users interacting with webpages or links for keywords, the system comprising:

• a memory storing instructions;
• and
• a processor executing the instructions to perform a process including:
• obtaining observations of value with respect to position for the keyword;
• receiving a Gaussian Process Model configured to obtain a prediction function and associated uncertainties;
• applying the Gaussian Process Model on the observations to obtain the prediction function and associated uncertainties, the prediction function and the associated uncertainties relating positions to expected values;
• receiving a Thompson sampling reinforcement learning model configured to obtain a target position;
• applying h Thompson sampling reinforcement learning model on the expected values and the positions to obtain h target position, the applying the Thompson sampling reinforcement learning model including an agent taking one of one or more action(s) from an action function according to a policy, the one or more action(s) including an exploit action or an explore action, the exploit action or the explore action being chosen according to an exploit-explore ratio of the policy, the policy being updated according to a history of actions, states, and rewards, the rewards corresponding to the values associated with the keyword;
• obtaining historical bid position data including a value-position-time set for the keyword, the value-position-time set including a plurality of value-position-time triplets for the keyword;
• applying a backwards filtering model to the historical bid position data to form a bid-to-position function by starting with a most recent value-position-time triplet of the plurality of value-position-time triplets and progressing backwards in time to form a monotonically decreasing function of values and positions of the value-position-time set selected according to a valid-or-not algorithm;
• determining bid information using the target position and the bid-to-position function;
• and
• transmitting a bid message to a search engine, the bid message including the bid information.

A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform a method for targeting bid and position for a keyword by tracking users interacting with webpages or links for keywords, the method comprising:

• obtaining observations of value with respect to position for the keyword, the observations being based at least on keyword value information, the keyword value information being associated with the keyword based on user device tracking and/or user(s) selecting webpages or links that were displayed at specific positions of lists for the keyword;
• receiving a Gaussian Process Model configured to obtain a prediction function and associated uncertainties;
• applying the Gaussian Process Model on the observations to obtain the prediction function and associated uncertainties, the prediction function and the associated uncertainties relating positions to expected values;
• receiving a Thompson sampling reinforcement learning model configured to obtain a target position;
• applying the Thompson sampling reinforcement learning model on the expected values and the positions to obtain a target position the applying the Thompson sampling reinforcement learning model including an agent taking one of one or more action(s) from an action function according to a policy, the one or more action(s) including an exploit action or an explore action, the exploit action or the explore action being chosen according to an exploit-explore ratio of the policy, the policy being updated according to a history of actions, states, and rewards, the rewards corresponding to the values associated with the keyword;
• obtaining historical bid position data including a value-position-time set for the keyword;
• applying a backwards filtering model to the historical bid position data to form a bid-to-position function;
• determining bid information using the target position and the bid-to-position function;
• and
• transmitting a bid message to a search engine, the bid message including the bid information.