Systems and methods for descriptive machine learning based underwriting

A non-transitory computer readable storage medium including instructions that, when executed by a computer processing system, cause the computer processing system to perform operations for generating a decision based on document images, the operations comprising:

• receiving a document image corresponding to a form and data depicted within the document image of the form for which the decision is to be rendered from the data;
• performing a first machine learning based analysis of the document image by transforming the document image, using a first machine learning model trained to digitize the form from the document image, the first machine learning model digitizing the document image and extracting one or more keys and one or more values corresponding to the keys from a digitized version of the document image, wherein a sequential pipeline of at least two or more trained machine learning based image analysis processes perform the digitizing of the document image and the extracting the one or more keys and one or more values to transform the document image;
• performing a second machine learning based analysis using the one or more keys and one or more values corresponding to keys as an input into a second machine learning model that generates the decision based on key, value pairs, the second machine learning model, the second machine learning model comprises a tree based model trained to generate the decision based on key, value pairs, and generating the decision using the tree based model further comprises:
• generating the decision based on an analysis of the key, value pairs by the tree based model, wherein the tree based model comprises a plurality of splits, and the generation of the decision further comprises;
• tracking splits made in the tree based model based on an evaluation of the key, value pairs at each split in the plurality of splits, and tracking a flow of decisions and the associated key, value pair used in each of the decisions through the tree based model,
• generating an explanation of how the second machine learning model generated the decision, the explanation comprising:
• (i) a decision explainability graph, (ii) a confidence of the decision based on the tree based model, (iii) feature importance of two or more keys used by the tree based model in arriving at the decision, and (iv) a verbose explanation of the decision, and generating the explanation comprises:
• modeling the flow of decisions made within the tree based decision model as a directed graph, wherein a collection of decisions comprised in the directed graph provides the verbose explanation as individual decisions that contributed to the decision,
• mapping the tracked flow of individual decisions performed by the tree based model including tracking splits in the tree based model based on associated key, value pair evaluated at each split,
• generating the decision explainability graph from the mapping, the decision explainability graph comprising a directed graph with nodes, edges, and a weight assigned to each edge, wherein the decision explainability graph comprises a start node of the directed graph representing a transition to an initial split, at least one decision node representing an intermediate decision within the directed graph, and a plurality of end nodes of the directed graph representing the decision, and wherein each edge joins two nodes and comprises a weighted direction corresponding to a number of tracked transitions in the tree based decision model between the two nodes in a given direction when making the decisions, and
• generating, from the decision explainability graph, the feature importance for the two or more keys used by the tree based decision model in arriving at the decision, wherein the feature importance comprises an inferred relative importance a key, value pair from the plurality of key, value pairs in influencing the decision, wherein each key and associated value from the key, value pairs is assigned a feature importance value based the generated decision graph,
• transmitting, to an organization system, a user interface generated using the decision, the confidence of the decision determined from the second machine learning based analysis, and the explanation of how the second machine learning model generated the decision, further comprising:
• detecting when the confidence of the decision generated by the second machine learning model fails to satisfy a decision confidence threshold,
• rendering, in the user interface, (i) data indicating a failure to satisfy the decision confidence threshold and one or more of the keys (ii) associated values extracted from the document image and the document image (iii) explainability data comprising the explainability graph, the verbose explanation, and two or more keys and their corresponding importance values, the explainability data indicative of the explanation of how the second machine learning model generated the decision, and (iv) the confidence of the decision determined from the second machine learning based analysis,
• receiving a user decision from the organization system that changes one or more of the keys extracted from the document image, associated values extracted from the document image, key-value pairs, and a revised confidence value of the decision,
• storing the user decision along with the one or more of the keys and associated values extracted from the document image, the revised confidence value, and the document image in a collection of training data for retraining the second machine learning model, and
• periodically performing retraining of the second machine learning model based at least in part on the collection of training data comprising a plurality of user decisions to generated a retrained second machine learning model;
• and
• executing the first machine learning model for extracting one or more second keys and one or more second values corresponding to keys from a digitized version of a second document image, and executing the retrained second machine learning model to generate a second decision using the extracted one or more second keys and associated one or more second values.

A system, comprising:

• a network interface configured to receive a document image corresponding to a form and data depicted within the document image of the form for which decision is to be rendered from the data;
• a memory that stores the document image;
• and
• a processor coupled with the memory configured to access the document image and further configured to:
• perform a first machine learning based analysis of the document image by transforming the document image, using a first machine learning model trained to digitize the form from the document image, the first machine learning model digitizing the document image and extracting one or more keys and one or more values corresponding to the keys from a digitized version of the document image, wherein a sequential pipeline of at least two or more trained machine learning based image analysis processes perform the digitizing of the document image and the extracting the one or more keys and one or more values to transform the document image,
• perform a second machine learning based analysis using the one or more keys and one or more values corresponding to keys as an input into a second machine learning model that generates the decision based on key, value pairs, the second machine learning model, the second machine learning model comprises a tree based model trained to generate the decision based on key, value pairs, and the processor configured to generate the decision using the tree based model further comprises the processor configured to:
• generate the decision based on an analysis of the key, value pairs by the tree based model, wherein the tree based model comprises a plurality of splits, and the generation of the decision further comprises:
• tracking splits made in the tree based model based on an evaluation of the key, value pairs at each split in the plurality of splits, and tracking a flow of decisions and the associated key, value pair used in each of the decisions through the tree based model,
• generate an explanation of how the second machine learning model generated the decision, the explanation comprising:
• (i) a decision explainability graph, (ii) a confidence of the decision based on the tree based model, (iii) feature importance of two or more keys used by the tree based model in arriving at the decision, and (iv) a verbose explanation of the decision, and generation the explanation comprises the processor configured to:
• model the flow of decisions made within the tree based decision model as a directed graph, wherein a collection of decisions comprised in the directed graph provides the verbose explanation as individual decisions that contributed to the decision,
• map the tracked flow of individual decision performed by the tree based model including tracking splits in the tree based model based on associated key, value pair evaluated at each split,
• generate the decision explainability graph from the mapping, the decision explainability graph comprising a directed graph with nodes, edges, and a weight assigned to each edge, wherein the decision explainability graph comprises a start node of the directed graph representing a transition to an initial split, at least one decision node representing an intermediate decision within the directed graph, and a plurality of end nodes of the directed graph representing the decision, and wherein each edge joins two nodes and comprises a weighted direction corresponding to a number of tracked transitions in the tree based decision model between the two nodes in a given direction when making the decisions, and
• generate, from the decision explainability graph, the feature importance for the two or more keys used by the tree based decision model in arriving at the decision, wherein the feature importance comprises an inferred related importance a key, value pair from the plurality of key, value pairs in influencing the decision, wherein each key and associated value from the key, value pairs is assigned a feature importance value based on the generated decision graph,
• transmit, to an organization system, a user interface generated using the decision, the confidence of the decision determined from the second machine learning based analysis, and the explanation of how the second machine learning model generated the decision, and wherein the processor is further configured to:
• detect when the confidence of the decision generated by the second machine learning model fails to satisfy a decision confidence threshold,
• render, in the user interface, (i) data indicating a failure to satisfy the decision confidence threshold and one or more of the keys, (ii) associated values extracted from the document image and the document image, (iii) explainability data comprising the explainability graph, the verbose explanation, and two or more keys and their corresponding importance values, the explainability data indicative of the explanation of how the second machine learning model generated the decision, and (iv) the confidence of the decision determined from the second machine learning based analysis,
• receive a user decision from the organization system that changes one or more of the keys extracted from the document image, associated values extracted from the document image, key-value pairs, and a revised confidence value of the decision,
• store the user decision along with the one or more of the keys and associated values extracted from the document image, the revised confidence value, and the document image in a collection of training data for retraining the second machine learning model, and
• periodically perform retraining of the second machine learning model based at least in part on the collection of training data comprising a plurality of user decisions to generate a retrained second machine learning model, and
• execute the first machine learning model for extracting one or more second keys and one or more second values corresponding to keys from a digitized version of a second document image, and executing the retrained second machine learning model to generate a second decision using the extracted one or more second keys and associated one or more second values.