# US 12,008,590 B2 · Machine learning model trained to predict conversions for determining lost conversions caused by restrictions in available fulfillment windows or fulfillment cost > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,008,590 B2 - Document ID: US12008590B2 - Status: Granted - Kind: B2 - Application number: 18/149,646 - Filed: 2023-01-03 - Published: 2023-05-18 - Granted: 2024-06-11 - Earliest claimed: 2021-08-16 - Assignees: Maplebear Inc. - Applicants: Maplebear Inc. - Inventors: Wa Yuan; Ganesh Krishnan; Qianyi Hu; Aishwarya Balachander; George Ruan; Soren Zeliger; Mike Freimer; Aman Jain - Web page: https://patentgrep.com/patent/US12008590B2 ## CPC - G06Q 30/0202: Market predictions or forecasting for commercial activities - G06N 3/084: Backpropagation, e.g. using gradient descent - G06N 5/01: Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound - G06N 20/20: Ensemble learning - G06Q 10/06312: {Adjustment or analysis of established resource schedule, e.g. resource or task levelling, or dynamic rescheduling} - G06Q 10/087: Inventory or stock management, e.g. order filling, procurement or balancing against orders - G06Q 10/0877: {by inventory control or reporting using inventory tracking or counting} - G06Q 30/0201: Market modelling; Market analysis; Collecting market data - G06Q 30/0607: {Regulating the sale of restricted items, e.g. alcohol} - G06Q 30/0633: {Managing shopping lists, e.g. compiling or processing purchase lists (shipping orders G06Q10/083; order filling G06Q10/087)} - G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract An online concierge system trains a machine learning conversion model that predicts a probability of receiving an order from a user when the user accesses the online concierge system. The conversion model predicts the probability of receiving the order based on a set of input features that include price and availability information. For each access to the online concierge system, the online concierge system applies the conversion model to a current price and availability and to an optimal price availability. The online concierge system generates a metric as the difference between the two predicted probabilities of receiving an order. ## Claims ### Claim 1 A computer program product comprising a non-transitory computer readable storage medium having instructions encoded thereon that, when executed by a processor, cause the processor to: retrieve a conversion prediction model that was generated by: obtaining training data comprising a plurality of examples, each example comprising a set of input features including discrete time intervals available for fulfilling an order, a price for fulfilling the order during each discrete time interval available for fulfilling the order, and a label applied to each example indicating whether an online concierge system received an order; initializing a conversion network that comprises a plurality of layers of a neural network, where the conversion network is configured to: receive a set of features including discrete time intervals available for fulfilling the order and a price for fulfilling the order during each discrete time interval available for fulfilling the order, and generate a predicted probability of the online concierge system receiving an order; and for each of a plurality of the examples of the training data: applying the conversion network to the received set of features; backpropagating one or more error terms obtained from one or more loss functions to update a set of parameters of the conversion network, the backpropagating performed through the conversion network and one or more of the error terms based on a difference between a label applied to the example and the predicted probability of the online concierge system receiving the order; and stopping the backpropagation after the one or more loss functions satisfy one or more criteria; receive an access to the online concierge system from a user during a particular discrete time interval; apply the conversion prediction model to a set of features describing the access to the online concierge system to predict a probability of the online concierge system receiving an order from the user during the particular discrete time interval; compare the predicted probability with a reference probability to determine a metric, indicating a likelihood of a value of compensation for fulfilling the order affecting the probability of the online concierge system receiving the order from the user during the particular discrete time interval; and modify the value of compensation for fulfilling the order based on the metric. ### Claim 2 (depends on 1) The computer program product of claim 1, the non-transitory computer readable storage medium having additional instructions encoded thereon that, when executed by a processor, cause the processor to: identify a subset set of input features from the set of features; and apply the conversion prediction model to the subset of features. ### Claim 3 (depends on 2) The computer program product of claim 2, wherein the subset of input features accounts for a geographic region of the access and a time of the access. ### Claim 4 (depends on 2) The computer program product of claim 2, wherein the subset set of input features identifies one or more discrete time intervals as unavailable for fulfilling the order based on a geographic region of the access or a time of the access. ### Claim 5 (depends on 4) The computer program product of claim 4, wherein the subset of input features comprises an optimal set of input features that identifies one or more discrete time intervals occurring after a closing time of a warehouse identified by the access as unavailable for fulfilling the order. ### Claim 6 (depends on 5) The computer program product of claim 5, wherein the optimal set of input features identifies one or more discrete time intervals as unavailable for fulfilling the order based on restrictions for purchasing one or more items for the geographic region of the access. ### Claim 7 (depends on 1) The computer program product of claim 1, wherein the set of input features comprises an availability input vector including different dimensions corresponding to different discrete time intervals for fulfilling the order and a value for a dimension indicating whether a discrete time interval corresponding to the dimension is available for fulfilling the order. ### Claim 8 (depends on 1) The computer program product of claim 1, wherein the set of input features includes a price input vector having different dimensions corresponding to different discrete time intervals, with a value of a dimension specifying a price for fulfilling the order during a discrete time interval corresponding to the dimension. ### Claim 9 (depends on 1) The computer program product of claim 1, wherein the set of input features further includes: a length of time a user accessed the online concierge system, a number of order the online concierge system received from the user, an amount of content the user viewed through the online concierge system, and any combination thereof. ### Claim 10 A computer implemented method, the method comprising: retrieving a conversion prediction model that was generated by: obtaining training data comprising a plurality of examples, each example comprising a set of input features including discrete time intervals available for fulfilling an order, a price for fulfilling the order during each discrete time interval available for fulfilling the order, and a label applied to each example indicating whether an online concierge system received an order; initializing a conversion network that comprises a plurality of layers of a neural network, where the conversion network is configured to: receive a set of features including discrete time intervals available for fulfilling the order and a price for fulfilling the order during each discrete time interval available for fulfilling the order, and generate a predicted probability of the online concierge system receiving an order; and for each of a plurality of the examples of the training data: applying the conversion network to the received set of features; backpropagating one or more error terms obtained from one or more loss functions to update a set of parameters of the conversion network, the backpropagating performed through the conversion network and one or more of the error terms based on a difference between a label applied to the example and the predicted probability of the online concierge system receiving the order; and stopping the backpropagation after the one or more loss functions satisfy one or more criteria; receiving an access to the online concierge system from a user during a particular discrete time interval; applying the conversion prediction model to a set of features describing the access to the online concierge system to predict a probability of the online concierge system receiving an order from the user during the particular discrete time interval; comparing the predicted probability with a reference probability to determine a metric, indicating a likelihood of a value of compensation for fulfilling the order affecting the probability of the online concierge system receiving the order from the user during the particular discrete time interval; and modifying the value of compensation for fulfilling the order based on the metric. ### Claim 11 (depends on 10) The method of claim 10, wherein the method further comprises: identifying a subset set of input features from the set of features; and applying the conversion prediction model to the subset of features. ### Claim 12 (depends on 11) The method of claim 11, wherein the subset of input features comprises an optimal set of input features from the set of features, and the optimal set of input features accounts for a geographic region of the access and a time of the access. ### Claim 13 (depends on 12) The method of claim 12, wherein the optimal set of input features identifies one or more discrete time intervals as unavailable for fulfilling the order based on the geographic region of the access or a time of the access. ### Claim 14 (depends on 12) The method of claim 12, wherein the optimal set of input features identifies one or more discrete time intervals occurring after a closing time of a warehouse identified by the access as unavailable for fulfilling the order. ### Claim 15 (depends on 12) The method of claim 12, wherein the optimal set of input features identifies one or more discrete time intervals as unavailable for fulfilling the order based on restrictions for purchasing one or more items for the geographic region of the access. ### Claim 16 (depends on 10) The method of claim 10, wherein the set of input features comprises an availability input vector including different dimensions corresponding to different discrete time intervals for fulfilling the order and a value for a dimension indicating whether a discrete time interval corresponding to the dimension is available for fulfilling the order. ### Claim 17 (depends on 10) The method of claim 10, wherein the set of input features includes a price input vector having different dimensions corresponding to different discrete time intervals, with a value of a dimension specifying a price for fulfilling the order during a discrete time interval corresponding to the dimension. ### Claim 18 (depends on 10) The method of claim 10, wherein the set of input features further includes: a length of time a user accessed the online concierge system, a number of order the online concierge system received from the user, an amount of content the user viewed through the online concierge system, and any combination thereof. ### Claim 19 A computer system, comprising: a processor; and a non-transitory computer readable storage medium having instructions encoded thereon that, when executed by the processor, cause the processor to: retrieve a conversion prediction model that was generated by: obtaining training data comprising a plurality of examples, each example comprising a set of input features including discrete time intervals available for fulfilling an order, a price for fulfilling the order during each discrete time interval available for fulfilling the order, and a label applied to each example indicating whether an online concierge system received an order; initializing a conversion network that comprises a plurality of layers of a neural network, where the conversion network is configured to: receive a set of features including discrete time intervals available for fulfilling the order and a price for fulfilling the order during each discrete time interval available for fulfilling the order, and generate a predicted probability of the online concierge system receiving an order; and for each of a plurality of the examples of the training data: applying the conversion network to the received set of features; backpropagating one or more error terms obtained from one or more loss functions to update a set of parameters of the conversion network, the backpropagating performed through the conversion network and one or more of the error terms based on a difference between a label applied to the example and the predicted probability of the online concierge system receiving the order; and stopping the backpropagation after the one or more loss functions satisfy one or more criteria; receive an access to the online concierge system from a user during a particular discrete time interval; apply the conversion prediction model to a set of features describing the access to the online concierge system to predict a probability of the online concierge system receiving an order from the user during the particular discrete time interval; compare the predicted probability with a reference probability to determine a metric, indicating a likelihood of a value of compensation for fulfilling the order affecting the probability of the online concierge system receiving the order from the user during the particular discrete time interval; and modify the value of compensation for fulfilling the order based on the metric. ### Claim 20 (depends on 19) The computer system of claim 19, the non-transitory computer readable storage medium having additional instructions encoded thereon that, when executed by a processor, cause the processor to: identify a subset set of input features from the set of features; and apply the conversion prediction model to the subset of features.