Methods and apparatuses for evaluating substitutions made during order fulfillment processes

A method comprising:

• receiving order data indicating an order placed by a customer on an e-commerce platform;
• routing the order into a test group or a control group when the order data indicates that the order will be filled by a store participating in a comparison test;
• applying test features to the order if the order was routed into the test group and apply control features to the order if the order was routed into the control group;
• determining recommended substitute items based on the test features or the control features, the recommended substitute items intended to replace items ordered by the customer that are unavailable;
• and
• determining one or more performance metrics of the test group and the control group.

A non-transitory computer readable medium having instructions stored thereon, wherein the instructions, when executed by at least one processor, cause a device to perform operations comprising:

• receiving order data indicating an order placed by a customer on an e-commerce platform;
• routing the order into a test group or a control group when the order data indicates that the order will be filled by a store participating in a comparison test;
• applying test features to the order if the order was routed into the test group and apply control features to the order if the order was routed into the control group;
• determining recommended substitute items based on the test features or the control features, the recommended substitute items intended to replace items ordered by the customer that are unavailable;
• and
• determining one or more performance metrics of the test group and the control group.