# US 2024/0202784 A1 · Identifying Feedback for a Picker for an Online Concierge System Affected by External Conditions

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0202784 A1
- Document ID: US20240202784A1
- Status: Published application
- Kind: A1
- Application number: 18/085,458
- Filed: 2022-12-20
- Published: 2024-06-20
- Earliest claimed: 2022-12-20
- Assignees: Maplebear, Inc.(dba Instacart)
- Applicants: Maplebear Inc. (dba Instacart)
- Inventors: Calder Hayes; Aakash Singh; Jaclyn Tandler; Alicia Wei; Gareth Pennington; Shengwen Fang
- Web page: https://patentgrep.com/patent/US20240202784A1

## CPC

- G06Q 30/0282: Rating or review of business operators or products
- G06Q 10/087: Inventory or stock management, e.g. order filling, procurement or balancing against orders
- G06Q 10/0877: {by inventory control or reporting using inventory tracking or counting}
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR

## Abstract

An online concierge system provides orders to a picker who fulfills the order by delivering items from the order to a customer. Customers provide feedback to the online concierge system about pickers, which is used when the online concierge system allocates orders to pickers. In some cases, negative feedback may be caused by conditions external to a picker, such as weather conditions or an inability to access the online concierge system. To avoid penalizing pickers because of external conditions, the online concierge system identifies forgiveness events when external conditions affect order fulfillment. Feedback affected by a forgiveness event is identified by the online concierge system when evaluating pickers. To facilitate matching of feedback to forgiveness events, a forgiveness event table indexes identified forgiveness events to reduce computing complexity and resources, while simplifying addition of new forgiveness events.

## Claims

### Claim 1

A method comprising, at a computer system comprising a processor and a computer-readable medium:
identifying a forgiveness event based on characteristics associated with an order, the order being received from a customer associated with a service provided by the computer system, the forgiveness event specifying one or more conditions affecting fulfillment of the order by a picker;
storing the forgiveness event as an entry in a forgiveness event table, the entry including information specifying a type of the forgiveness event, an index identifier for the forgiveness event, and a time interval associated with the forgiveness event;
selecting feedback previously received by the computer system from customers for one or more pickers that includes information matching the index identifier for the forgiveness event; and
storing an indication of the forgiveness event in association with the selected feedback; and
presenting a notification on at least one picker device associated with at least one picker for which the selected feedback was received in response to storing the indication, the notification indicating at least a portion of feedback previously received for the picker has been forgiven by the computer system.

### Claim 2 (depends on 1)

The method of claim 1, wherein identifying the forgiveness event based on characteristics associated with the order received from the customer of the computer system comprises:
determining that characteristics of a geographic region in which the order is fulfilled satisfy one or more conditions.

### Claim 3 (depends on 2)

The method of claim 2, wherein determining that characteristics of the geographic region in which the order is fulfilled satisfy one or more conditions comprises:
determining that weather conditions in the geographic region satisfy one or more criteria.

### Claim 4 (depends on 2)

The method of claim 2, wherein determining that characteristics of the geographic region in which the order is fulfilled satisfy one or more conditions comprises:
determining that at least a threshold amount of pickers in the geographic region are unable to access a client application associated with the computer system.

### Claim 5 (depends on 1)

The method of claim 1, wherein identifying the forgiveness event based on characteristics associated with the order comprises:
determining that an item included in the order has at least a threshold inventory at a retailer specified by the order.

### Claim 6 (depends on 1)

The method of claim 1, wherein identifying the forgiveness event based on characteristics associated with the order comprises:
determining that the order was not presented to one or more pickers until after a time for fulfillment specified by the order.

### Claim 7 (depends on 1)

The method of claim 1, wherein identifying the forgiveness event based on characteristics associated with the order comprises:
determining that the customer provided fraudulent information to the computer system by applying a trained fraud detection model to information the computer system received from the customer, the model trained by:
generating a set of training examples, each training example including information and having a label indicating whether the information is fraudulent;
applying the fraud detection model to each training example of the set to generate an output for the training example;
scoring the output for the training example using a loss function and the label of the training example; and
updating one or more parameters of the model by backpropagation based on the scoring.

### Claim 8 (depends on 1)

The method of claim 1, wherein identifying the forgiveness event based on characteristics associated with the order comprises:
determining that at least a threshold amount of feedback on one or more pickers received from the customer satisfies one or more criteria.

### Claim 9 (depends on 1)

The method of claim 1, wherein selecting feedback previously received by the computer system from customers for one or more pickers that includes information matching the index identifier for the forgiveness event comprises:
selecting feedback previously received from at least one customer including information matching the index identifier for the forgiveness event and associated with a time within the time interval associated with the forgiveness event.

### Claim 10 (depends on 1)

The method of claim 1, further comprising:
receiving additional feedback about one or more pickers from one or more customers; and
storing the indication of the forgiveness event in association with the additional feedback in response to determining the additional feedback includes information matching the index identifier of the forgiveness event.

### Claim 11 (depends on 1)

The method of claim 1, further comprising:
receiving one or more orders at the computer system for fulfillment;
selecting a set of pickers to whom one or more of the received orders is presented based on feedback associated with pickers, the selecting disregarding feedback associated with the indication; and
presenting one or more of the received orders to the selected set of pickers.

### Claim 12

A computer program product comprising a non-transitory computer readable storage medium having instructions encoded thereon that, when executed by a processor, cause the processor to perform steps comprising:
identifying a forgiveness event based on characteristics associated with an order received from a customer associated with a service provided by a computer system, the forgiveness event specifying one or more conditions affecting fulfillment of the order by a picker;
storing the forgiveness event as an entry in a forgiveness event table, the entry including a type of the forgiveness event, an index identifier for the forgiveness event, and a time interval associated with the forgiveness event;
selecting feedback previously received by the computer system from customers for one or more pickers that includes information matching the index identifier for the forgiveness event;
storing an indication of the forgiveness event in association with the selected feedback; and
presenting a notification on at least one picker device associated with at least one picker for which the selected feedback was received in response to storing the indication, the notification indicating at least a portion of feedback previously received for the picker has been forgiven by the computer system.

### Claim 13 (depends on 12)

The computer program product of claim 12, wherein identifying the forgiveness event based on characteristics associated with the order received from the customer of the computer system comprises:
determining that characteristics of a geographic region in which the order is fulfilled satisfy one or more conditions.

### Claim 14 (depends on 13)

The computer program product of claim 13, wherein determining characteristics of the geographic region in which the order is fulfilled satisfy one or more conditions comprises:
determining weather conditions in the geographic region satisfy one or more criteria.

### Claim 15 (depends on 13)

The computer program product of claim 13, wherein determining characteristics of the geographic region in which the order is fulfilled satisfy one or more conditions comprises:
determining at least a threshold amount of pickers in the geographic region are unable to access a client application associated with the computer system.

### Claim 16 (depends on 12)

The computer program product of claim 12, wherein identifying the forgiveness event based on characteristics associated with the order received from the customer of the computer system comprises:
determining an item included in the order has at least a threshold inventory at a retailer specified by the order.

### Claim 17 (depends on 12)

The computer program product of claim 12, wherein identifying the forgiveness event based on characteristics associated with the order received from a user of the computer system comprises:
determining the order was not identified to one or more pickers until after a time for fulfillment specified by the order.

### Claim 18 (depends on 12)

The computer program product of claim 12, wherein identifying the forgiveness event based on characteristics associated with the order received from a user of the computer system comprises:
determining the customer provided fraudulent information to the computer system by applying a trained fraud detection model to information the computer system received from the customer, the model trained by:
generating a set of training examples, each training example including information and having a label indicating whether the information is fraudulent;
applying the fraud detection model to each training example of the set to generate an output for the training example;
scoring the output for the training example using a loss function and the label of the training example; and
updating one or more parameters of the model by backpropagation based on the scoring.

### Claim 19 (depends on 12)

The computer program product of claim 12, wherein non-transitory computer readable storage medium further has instructions encoded thereon that, when executed by the processor, cause the processor to perform steps comprising:
receiving additional feedback about one or more pickers from one or more customers; and
storing the indication in association with the additional feedback in response to determining the additional feedback includes information matching the index identifier of the forgiveness event.

### Claim 20

A system comprising:
one or more processors; and
a non-transitory computer readable storage medium having instructions encoded thereon that, when executed by the one or more processors, cause the system to perform steps comprising:
identifying a forgiveness event based on characteristics associated with an order received from a customer associated with a service provided by a computer system, the forgiveness event specifying one or more conditions affecting fulfillment of the order by a picker;
storing the forgiveness event as an entry in a forgiveness event table, the entry including a type of the forgiveness event, an index identifier for the forgiveness event, and a time interval associated with the forgiveness event;
selecting feedback previously received by the computer system from customers for one or more pickers that includes information matching the index identifier for the forgiveness event; and
storing an indication of the forgiveness event in association with the selected feedback; and
presenting a notification on at least one picker device associated with at least one picker for which the selected feedback was received in response to storing the indication, the notification indicating at least a portion of feedback previously received for the picker has been forgiven by the computer system.
