# US 2021/0241115 A1 · Techniques to perform global attribution mappings to provide insights in neural networks > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0241115 A1 - Document ID: US20210241115A1 - Status: Published application - Kind: A1 - Application number: 16/855,685 - Filed: 2020-04-22 - Published: 2021-08-05 - Earliest claimed: 2020-02-05 - Assignees: Capital One Services, LLC - Applicants: Capital One Services, LLC - Inventors: Mark IBRAHIM; John PAISLEY; Ceena MODARRES; Melissa LOUIE - Web page: https://patentgrep.com/patent/US20210241115A1 ## CPC - G06V 10/763: {Non-hierarchical techniques, e.g. based on statistics of modelling distributions} - G06F 18/23213: with fixed number of clusters, e.g. K-means clustering - G06N 3/04: Architecture, e.g. interconnection topology - G06N 3/044: Recurrent networks, e.g. Hopfield networks - G06N 3/0499: Feedforward networks - G06N 3/082: modifying the architecture, e.g. adding, deleting or silencing nodes or connections - G06N 3/084: Backpropagation, e.g. using gradient descent - G06N 3/09: Supervised learning - G06N 5/01: Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound - G06N 20/20: Ensemble learning - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract Embodiments include techniques to determine a set of credit risk assessment data samples, generate local credit risk assessment attributions for the set of credit risk assessment samples, and normalize each local credit risk assessment attribution of the local credit risk assessment attributions. Further, embodiments techniques to compare each pair of normalized local credit risk assessment attributions and assign a rank distance thereto proportional to a degree of ranking differences between the pair of normalized local credit risk assessment attributions. The techniques also include applying a K-medoids clustering algorithm to generate clusters of the local risk assessment attributions, generating global attributions, and determining insights for the neural network based on the global attributions. ## Claims ### Claim 1 A computer-implemented method, comprising: determining, by processing circuitry and from a datastore, a set of credit risk assessment data samples; generating, by the processing circuitry, local credit risk assessment attributions for the set of credit risk assessment samples, and each local credit risk assessment attribution includes a rank and a weight of an association between a particular feature and a particular prediction predicted by a neural network; normalizing, by the processing circuitry, each local credit risk assessment attribution of the local credit risk assessment attributions; comparing, by the processing circuitry, each pair of normalized local credit risk assessment attributions, and assigning a rank distance thereto proportional to a degree of ranking differences between the pair of normalized local credit risk assessment attributions; applying, by the processing circuitry, a K-medoids clustering algorithm to generate clusters of the local risk assessment attributions, wherein the K-medoids clustering algorithm utilizes the ranking distances for each pair of normalized local credit risk assessment attributions to assign each local credit risk assessment attribution to one of one or more clusters, and each cluster having a medoid; generating, by the processing circuitry, global attributions, wherein each global attribution corresponds to one of the medoids of one of the clusters; and determining, by the processing circuitry, a bias in the neural network, one or more features to include in the neural network, or a combination thereof based on the global attributions. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, wherein the local credit risk assessment attributions are represented by vectors, and each vector corresponds with a particular local credit risk assessment attribution, and each local credit risk assessment attribution is normalized by normalizing each of the vectors. ### Claim 3 (depends on 2) The computer-implemented method of claim 2, wherein comparing each pair of normalized local credit risk assessment attributes comprises comparing vectors associated with each of the pair of normalized local credit risk assessment attributions. ### Claim 4 (depends on 1) The computer-implemented method of claim 1, wherein the assigning the rank distance is done by calculating the rank distance using a weighted Kendall's Tau rank distance algorithm. ### Claim 5 (depends on 1) The computer-implemented method of claim 1, wherein the assigning the rank distance is done by calculating the rank distance using a weighted Spearman's Rho squared rank distance algorithm. ### Claim 6 (depends on 1) The computer-implemented method of claim 1, wherein the K-medoid clustering algorithm to identify K local credit risk assessment attributions to minimize pairwise dissimilarity with in a particular cluster relative to a particular medoid. ### Claim 7 (depends on 6) The computer-implemented method of claim 6, wherein the K-medoid clustering algorithm to minimize pairwise dissimilarity by initializing K local credit risk assessment attributions selected as medoids randomly from a total number of the local credit risk assessment attributions, and performing a number of iterations of assigning and reassigning a remaining portion of the total number of the local credit risk assessment attributions to one of the medoids until convergence or a predetermined number of iterations is met. ### Claim 8 (depends on 7) The computer-implemented method of claim 7, wherein the K-medoid clustering algorithm to assign and reassign the local credit risk assessment attributions to a nearest one of the medoids based on rank distances, and update medoids of clusters by minimizing the pairwise dissimilarity. ### Claim 9 A non-transitory processor-readable medium embodying a credit risk assessment program for global attribution mapping (GAM), the program to cause a processor: generate local credit risk assessment attributions for a set of credit risk assessment samples; normalize each local credit risk assessment attribution of the local credit risk assessment attributions; compare each pair of normalized local credit risk assessment attributions, and assigning a rank distance thereto proportional to a degree of ranking differences between the pair of normalized local credit risk assessment attributions; apply a medoids clustering algorithm to assign each local credit risk assessment attribution to a cluster having a medoid which is closest to the local credit risk assessment attribution based on the ranking distances, and which updates each cluster medoid by minimizing pairwise dissimilarity within a same cluster, and repeating running of the medoids clustering algorithm to adjust the local credit risk assessment attributions into converged clusters representing sub-populations having similar attributes; and utilize medoids and sizes of the converged clusters, to derive GAM insight into one or more of the local credit risk assessment attributions, wherein the GAM insight comprises a bias in a neural network, one or more features to include in the neural network, or a combination thereof. ### Claim 10 (depends on 9) The computer-readable medium of claim 9, wherein the assigning the rank distance is done by calculating the rank distance using a weighted Kendall's Tau rank distance algorithm. ### Claim 11 (depends on 9) The computer-readable medium of claim 9, wherein the assigning the rank distance is done by calculating the rank distance using a weighted Spearman's Rho squared rank distance algorithm. ### Claim 12 (depends on 9) The computer-readable medium of claim 9, wherein the local credit risk assessment attributions are represented by vectors, and each vector corresponds with a particular local credit risk assessment attribution, and each local credit risk assessment attribution is normalized by normalizing each of the vectors. ### Claim 13 (depends on 12) The computer-readable medium of claim 12, wherein the processor to compare each pair of normalized local credit risk assessment attributes comprises comparing vectors associated with each of the pair of normalized local credit risk assessment attributions. ### Claim 14 (depends on 9) The computer-readable medium of claim 9, wherein the medoid clustering algorithm to identify K local credit risk assessment attributions to minimize pairwise dissimilarity with in a particular cluster relative to a particular medoid. ### Claim 15 (depends on 14) The computer-readable medium of claim 14, wherein the medoid clustering algorithm to minimize pairwise dissimilarity by initializing K local credit risk assessment attributions selected as medoids randomly from a total number of the local credit risk assessment attributions, and the processor to perform a number of iterations of assigning and reassigning a remaining portion of the total number of the local credit risk assessment attributions to one of the medoids until convergence or a predetermined number of iterations is met. ### Claim 16 (depends on 15) The computer-readable medium of claim 15, wherein the medoid clustering algorithm to assign and reassign the local credit risk assessment attributions to a nearest one of the medoids based on rank distances, and update medoids of clusters by minimizing the pairwise dissimilarity. ### Claim 17 A system to perform global attribution mapping (GAM), comprising: a storage system comprising a set of credit risk assessment data samples; memory; and one or more processors coupled with the memory and storage system, the one or more processors configured to: determine, from the storage system, the set of credit risk assessment data samples; generate local credit risk assessment attributions for the set of credit risk assessment samples, and each local credit risk assessment attribution includes a rank and a weight of an association between a particular feature and a particular prediction predicted by a neural network; normalize each local credit risk assessment attribution; compare each pair of normalized local credit risk assessment attributions; assign a rank distance thereto proportional to a degree of ranking differences between the pair of normalized local credit risk assessment attributions based on the comparison; apply clustering algorithm to generate clusters of the local risk assessment attributions, wherein the clustering algorithm utilizes the ranking distances for each pair of normalized local credit risk assessment attributions to assign each local credit risk assessment attribution to one of one or more clusters, and each cluster having a medoid; generate global attributions, wherein each global attribution corresponds to a medoid of one of the clusters; and determine a bias in the neural network, one or more features include in the neural network, or a combination thereof based on the global attributions. ### Claim 18 (depends on 17) The system of claim 17, wherein the clustering algorithm comprises a K-medoid clustering algorithm to identify K local credit risk assessment attributions to minimize pairwise dissimilarity with in a particular cluster relative to a particular medoid, wherein K represents a number of clusters. ### Claim 19 (depends on 18) The system of claim 18, wherein the K-medoid clustering algorithm to minimize pairwise dissimilarity by initializing K local credit risk assessment attributions selected as medoids randomly from a total number of the local credit risk assessment attributions, and the one or more processors to perform a number of iterations of assigning and reassigning a remaining portion of the total number of the local credit risk assessment attributions to one of the medoids until convergence or a predetermined number of iterations is met. ### Claim 20 (depends on 19) The system of claim 19, wherein the K-medoid clustering algorithm to assign and reassign the local credit risk assessment attributions to a nearest one of the medoids based on rank distances, and update medoids of clusters by minimizing the pairwise dissimilarity.