Automated decision analysis by model operational characteristic curves

A system for optimizing a predictive fraud detection model and automatically enacting reactive measures based thereon, the system comprising at least one server computer including at least one processor, the at least one server computer programmed and/or configured to:

• receive transaction data representative of a plurality of transaction requests between at least one financial device holder and at least one merchant in a first time period, the transaction data comprising for each transaction request of the plurality of transaction requests at least one of the following parameters:
• transaction date, transaction time, transaction value, merchant type, transaction type, merchant location, or any combination thereof;
• receive, from a user interface, an identification of the predictive fraud detection model to be evaluated;
• in response to receiving the identification, receive the predictive fraud detection model to be evaluated, the predictive fraud detection model configured to categorize each transaction request as fraudulent or legitimate based at least partially on the parameters of the transaction request, the predictive fraud detection model further configured to order the plurality of transaction requests, relative to each other transaction request, from most likely fraudulent to least likely fraudulent;
• generate a performance evaluation dataset comprising an output of the predictive fraud detection model given an input of the transaction data;
• generate plotting data configured to cause a visual display of the user interface to represent at least two model performance metrics of the performance evaluation dataset on a same output plot, the same output plot having an x-axis of percentile threshold for transaction rejection and a y-axis of percent of metric value of the at least two model performance metrics, the at least two model performance metrics calculated from the performance evaluation dataset, and the at least two model performance metrics being a false positive ratio and at least one of the following:
• detection rate, maximum detectable rate, minimum false positive ratio, model performance score, random model false positive ratio, random model detection rate, or any combination thereof;
• transmit the plotting data to the user interface, wherein the plotting data is further configured to, in response to input in the user interface comprising a specification of a quantitative value of percentile threshold for transaction rejection, generate a corresponding metric value of each of the at least two model performance metrics in the same output plot;
• receive a user input comprising the specification of the quantitative value of percentile threshold for transaction rejection;
• and
• in response to receiving the user input, automatically reject a top percent of the plurality of transaction requests for suspected fraudulent transaction activity, the top percent determined at least partially from a customized rejection algorithm based on the user input.

A computer program product for optimizing a predictive fraud detection model and automatically enacting reactive measures based thereon, the computer program product comprising at least one non-transitory computer-readable medium including program instructions that, when executed by at least one processor, cause the at least one processor to:

• receive transaction data representative of a plurality of transaction requests between at least one financial device holder and at least one merchant in a first time period, the transaction data comprising for each transaction request of the plurality of transaction requests at least one of the following parameters:
• transaction date, transaction time, transaction value, merchant type, transaction type, merchant location, or any combination thereof;
• receive, from a user interface, an identification of the predictive fraud detection model to be evaluated;
• in response to receiving the identification, receive the predictive fraud detection model to be evaluated, the predictive fraud detection model configured to categorize each transaction request as fraudulent or legitimate based at least partially on the parameters of the transaction request, the predictive fraud detection model further configured to order the plurality of transaction requests, relative to each other transaction request, from most likely fraudulent to least likely fraudulent;
• generate a performance evaluation dataset comprising an output of the predictive fraud detection model given an input of the transaction data;
• generate plotting data configured to cause a visual display of the user interface to represent at least two model performance metrics of the performance evaluation dataset on a same output plot, the same output plot having an x-axis of percentile threshold for transaction rejection and a y-axis of percent of metric value of the at least two model performance metrics, the at least two model performance metrics calculated from the performance evaluation dataset, and the at least two model performance metrics being a false positive ratio and at least one of the following:
• detection rate, maximum detectable rate, minimum false positive ratio, model performance score, random model false positive ratio, random model detection rate, or any combination thereof;
• transmit the plotting data to the user interface, wherein the plotting data is further configured to, in response to input in the user interface comprising a specification of a quantitative value of percentile threshold for transaction rejection, generate a corresponding metric value of each of the at least two model performance metrics in the same output plot;
• receive a user input comprising the specification of the quantitative value of percentile threshold for transaction rejection;
• and
• in response to receiving the user input, automatically reject a top percent of the plurality of transaction requests for suspected fraudulent transaction activity, the top percent determined at least partially from a customized rejection algorithm based on the user input.