US 11,531,887 B1Granted

Disruptive prediction with ordered treatment candidate bins

AssigneeAmazon Technologies, Inc.

InventorsNaveen Sudhakaran Nair, Pragyana K. Mishra

Life of this patent

Estimating the term…

Loading patent…

Abstract

Prediction of outcomes of disruptive treatments are enabled utilizing sequenced training of a machine learning model over ordered bins of treatment candidates. Treatment candidates may be assigned to candidate characterization bins with an ordering, and the model may be trained with a sequence of training steps corresponding to the ordering of the candidate characterization bins, in each training step the model having untreated candidate features from a corresponding bin and aggregate metrics from one or more previous steps as input. The predicted outcome for a selected bin may be generated with the trained model having treated candidate features and aggregate metrics from one or more previous steps as input. The predicted outcome may be a counterfactual prediction for a bin with insufficient control candidates, and may represent a nonlinear extrapolation from control data in prior bins in the bin ordering.

Claims20

Dependent claims

A computer-implemented method, comprising:

determining, by a computer system, a propensity score for individual users of a plurality of users, the plurality of users being candidates for a treatment;
partitioning, by the computer system, the plurality of users into a plurality of propensity bins based at least in part on the propensity score determined for individual users of the plurality of users, individual propensity bins of the plurality of propensity bins being associated with a propensity score range;
training, by the computer system, a machine learning model for a selected propensity bin of the plurality of propensity bins at least in part by:
arranging, by the computer system, the plurality of propensity bins into a sequence of propensity bins based at least in part on the propensity score range associated with individual propensity bins of the plurality of propensity bins;
and
training, by the computer system, the machine learning model in a sequence of training operations corresponding to the sequence of propensity bins, in individual training operations of the sequence of training operations using observed outcomes of untreated users in the corresponding propensity bin to update the machine learning model given input including (i) features of the untreated users and (ii) an aggregate metric on predicted outcomes generated by the machine learning model in a prior training operation of the sequence of training operations;
generating, by the computer system, a counterfactual prediction for treatment of users in the selected propensity bin using the trained machine learning model given input including (i) features of treated users of the selected propensity bin and (ii) an aggregate metric on predicted outcomes generated by the machine learning model in a training operation of the sequence of training operations;
and
providing, by the computer system, a representation of the counterfactual prediction for presentation with a user interface.

6 more elements

A computer system, comprising:

a processor;
a memory storing computer-executable instructions that, when executed with the processor, cause the computer system to, at least:
assign a plurality of treatment candidates to a plurality of candidate characterization bins based at least in part on a candidate characterization score determined for individual treatment candidates of the plurality of treatment candidates;
train a machine learning model in a sequence of training operations corresponding to a ordering of candidate characterization bins, in individual training operations of the sequence of training operations using observed outcomes of untreated treatment candidates in the corresponding candidate characterization bin to update the machine learning model given input including (i) features of the untreated treatment candidates and (ii) an aggregate metric on predicted outcomes generated by the machine learning model in a prior training operation of the sequence of training operations;
generate a predicted outcome for treatment candidates in a selected candidate characterization bin using the trained machine learning model given input including (i) features of treated treatment candidates of the selected candidate characterization bin and (ii) an aggregate metric on predicted outcomes generated by the machine learning model in a training operation of the sequence of training operations;
and
provide a representation of the predicted outcome for presentation with a user interface.

4 more elements

Dependencies are inferred from the claim text.