Back to results
US 11,443,335 B2Granted
Model-based deep reinforcement learning for dynamic pricing in an online ride-hailing platform
AssigneeBeijing Didi Infinity Technology and Development Co., Ltd.
InventorXiaocheng Tang
Life of this patent
Estimating the term…
Abstract
Dynamic pricing may be applied in an online ride-hailing platform. Information may be obtained. The information may include a set of pricing candidates and an initial status of a ride-hailing platform. The set of pricing candidates may be updated based on the initial status of the ride-hailing platform to minimize a cross-entropy between the set of pricing candidates and a target pricing policy that maximizes a total income of the ride-hailing platform. A price for at least one current trip request on the ride-hailing platform may be generated based on the updated set of pricing candidates.
Claims16
A computer-implemented method for dynamic pricing, comprising:
- training a reinforcement learning (RL) model in a simulation of a ride-hailing platform to predict a future state of the simulation based on a previous state of the simulation and a previous pricing action, wherein the training comprises minimizing a divergence between the predicted future state of the simulation and an observed futured state of the simulation, and the simulation is an offline version of the ride-hailing platform;
- obtaining information comprising a set of pricing candidates and an initial status of the ride-hailing platform, wherein the initial status comprises a number of available drivers and a number of trip requests from passengers of the ride-hailing platform;
- updating the set of pricing candidates based on the initial status of the ride-hailing platform to minimize a cross-entropy between the set of pricing candidates and a target pricing policy that maximizes a total income of the ride-hailing platform, wherein the updating comprises:
- inputting the obtained information into the trained RL model;
- obtaining a cumulative reward from the trained RL model for each pricing candidate in the set of pricing candidates based on the initial status of the ride-hailing platform;
- obtaining a set of top pricing candidates comprising one or more pricing candidates with the highest cumulative rewards;
- and
- updating the set of pricing candidates based on the set of top pricing candidates;
- and
- generating a price for at least one current trip request on the ride-hailing platform based on the updated set of pricing candidates;
- collecting an observed status of the ride-hailing platform after the generated price is deployed online;
- retraining the RL model to reach a convergence between the prediction of the future status of the ride-hailing platform and the observed status of the ride-hailing platform.
9 more elements
A system for dynamic pricing, comprising one or more processors and one or more non-transitory computer-readable memories coupled to the one or more processors and configured with instructions executable by the one or more processors to cause the system to perform operations comprising:
- training a reinforcement learning (RL) model in a simulation of a ride-hailing platform to predict a future state of the simulation based on a previous state of the simulation and a previous pricing action, wherein the training comprises minimizing a divergence between the predicted future state of the simulation and an observed futured state of the simulation;
- obtaining information comprising a set of pricing candidates and an initial status of the ride-hailing platform, wherein the initial status comprises a number of available drivers and a number of trip requests from passengers of the ride-hailing platform;
- updating the set of pricing candidates based on the initial status of the ride-hailing platform to minimize a cross-entropy between the set of pricing candidates and a target pricing policy that maximizes a total income of the ride-hailing platform, wherein the updating comprises:
- inputting the obtained information into the trained RL model;
- obtaining a cumulative reward from the trained RL model for each pricing candidate in the set of pricing candidates based on the initial status of the ride-hailing platform;
- obtaining a set of top pricing candidates comprising one or more pricing candidates with the highest cumulative rewards;
- and
- updating the set of pricing candidates based on the set of top pricing candidates;
- and
- generating a price for at least one current trip request on the ride-hailing platform based on the updated set of pricing candidates;
- collecting an observed status of the ride-hailing platform after the generated price is deployed;
- retraining the RL model to reach a convergence between the prediction of the future status of the ride-hailing platform and the observed status of the ride-hailing platform.
9 more elements
A non-transitory computer-readable storage medium configured with instructions executable by one or more processors to cause the one or more processors to perform operations comprising:
- training a reinforcement learning (RL) model in a simulation of a ride-hailing platform to predict a future state of the simulation based on a previous state of the simulation and a previous pricing action, wherein the training comprises minimizing a divergence between the predicted future state of the simulation and an observed futured state of the simulation;
- obtaining information comprising a set of pricing candidates and an initial status of the ride-hailing platform, wherein the initial status comprises a number of available drivers and a number of trip requests from passengers of the ride-hailing platform;
- updating the set of pricing candidates based on the initial status of the ride-hailing platform to minimize a cross-entropy between the set of pricing candidates and a target pricing policy that maximizes a total income of the ride-hailing platform, wherein the updating comprises:
- inputting the obtained information into the trained RL model;
- obtaining a cumulative reward from the trained RL model for each pricing candidate in the set of pricing candidates based on the initial status of the ride-hailing platform;
- obtaining a set of top pricing candidates comprising one or more pricing candidates with the highest cumulative rewards;
- and
- updating the set of pricing candidates based on the set of top pricing candidates;
- and
- generating a price for at least one current trip request on the ride-hailing platform based on the updated set of pricing candidates;
- collecting an observed status of the ride-hailing platform after the generated price is deployed;
- retraining the RL model to reach a convergence between the prediction of the future status of the ride-hailing platform and the observed status of the ride-hailing platform.
9 more elements
Dependencies are inferred from the claim text.