Machine learning model trained to predict conversions for determining lost conversions caused by restrictions in available fulfillment windows or fulfillment cost

A computer implemented method, the method comprising:

• retrieving a conversion prediction model that was generated by:
• obtaining training data comprising a plurality of examples, each example comprising a set of input features including discrete time intervals available for fulfilling an order, a price for fulfilling the order during each discrete time interval available for fulfilling the order, and a label applied to each example indicating whether an online concierge system received an order;
• initializing a conversion network that comprises a plurality of layers of a neural network, where the conversion network is configured to:
• receive a set of features including discrete time intervals available for fulfilling the order and a price for fulfilling the order during each discrete time interval available for fulfilling the order, and
• generate a predicted probability of the online concierge system receiving an order;
• and
• for each of a plurality of the examples of the training data:
• applying the conversion network to the received set of features;
• backpropagating one or more error terms obtained from one or more loss functions to update a set of parameters of the conversion network, the backpropagating performed through the conversion network and one or more of the error terms based on a difference between a label applied to the example and the predicted probability of the online concierge system receiving the order;
• and
• stopping the backpropagation after the one or more loss functions satisfy one or more criteria;
• receiving an access to the online concierge system from a user during a particular discrete time interval;
• applying the conversion prediction model to a set of features describing the access to the online concierge system to predict a probability of the online concierge system receiving an order from the user during the particular discrete time interval;
• comparing the predicted probability with a reference probability to determine a metric, indicating a likelihood of a value of compensation for fulfilling the order affecting the probability of the online concierge system receiving the order from the user during the particular discrete time interval;
• and
• modifying the value of compensation for fulfilling the order based on the metric.

A computer system, comprising:

• a processor;
• and
• a non-transitory computer readable storage medium having instructions encoded thereon that, when executed by the processor, cause the processor to:
• retrieve a conversion prediction model that was generated by:
• obtaining training data comprising a plurality of examples, each example comprising a set of input features including discrete time intervals available for fulfilling an order, a price for fulfilling the order during each discrete time interval available for fulfilling the order, and a label applied to each example indicating whether an online concierge system received an order;
• initializing a conversion network that comprises a plurality of layers of a neural network, where the conversion network is configured to:
• receive a set of features including discrete time intervals available for fulfilling the order and a price for fulfilling the order during each discrete time interval available for fulfilling the order, and
• generate a predicted probability of the online concierge system receiving an order;
• and
• for each of a plurality of the examples of the training data:
• applying the conversion network to the received set of features;
• backpropagating one or more error terms obtained from one or more loss functions to update a set of parameters of the conversion network, the backpropagating performed through the conversion network and one or more of the error terms based on a difference between a label applied to the example and the predicted probability of the online concierge system receiving the order;
• and
• stopping the backpropagation after the one or more loss functions satisfy one or more criteria;
• receive an access to the online concierge system from a user during a particular discrete time interval;
• apply the conversion prediction model to a set of features describing the access to the online concierge system to predict a probability of the online concierge system receiving an order from the user during the particular discrete time interval;
• compare the predicted probability with a reference probability to determine a metric, indicating a likelihood of a value of compensation for fulfilling the order affecting the probability of the online concierge system receiving the order from the user during the particular discrete time interval;
• and
• modify the value of compensation for fulfilling the order based on the metric.