US 2024/0177094 A1Published application

Automatic Alert Dispositioning using Artificial Intelligence

AssigneeBank of America Corporation

InventorsYvonne Li, Franklin Kaiyuen Chan, Min Kyung Kim, Jared Scott Ginsberg, Aaron Blogg

Life of this patent

Estimating the term…

Loading patent…

Abstract

The system identifies false positives about a user and closes resulting alerts before the alert wastefully consume system resources. Machine learning techniques including clustering and multi-labeling classification are used to effectively categorize prior text-based notes to efficiently identify and automatically close false positive alerts. Moreover, weak labeling, AI transformers/sentence transformation, and/or k-means cluster analysis provide a means for condensing large quantities of textual data into ML model components with improved interpretability. Customer relationship management (CRM) platform and Risk Management Supervision (RMS) note analysis captures inefficiently/ineffectively organized past work and leverages it to reduce redundancies in future expert user/supervisory/customer advisory efforts. The various ML techniques disclosed herein output numerical features that directly improve model performance for alert triaging as a whole. Streamlining the presentation of large textual data to reviewers/users (e.g., supervision principals) as individual sentences or numerical values greatly improves explainability as a byproduct.

Claims20

Dependent claims

A method for enhanced triaging of alerts generated for customer service advisors, to reduce false positive alerts using machine learning and dimensionality reduction before multi-label classification of textual notes about interactions between the customer service advisors and customers, the method comprising:

displaying, on a graphical user interface (GUI), an alert generated for a customer service advisor, the alert being for a customer with a likelihood of satisfying a first scenario from among an enumerated list of scenarios;
contemporaneous with servicing of the customer, augmenting the alert displayed on the GUI with a visual indicator of a likelihood that the alert corresponding to the customer is false positive based on a machine learning (ML) model;
receiving, through the GUI, feedback provided by the customer service advisor about accuracy of the likelihood that the alert is false positive;
iteratively updating, based on the feedback, the ML model, wherein the ML model:
infers a likelihood that the customer was previously notified of the first scenario by clustering numerical representations of the textual notes, which represent interactions between the customer service advisor and the customer, in relation to the enumerated list of scenarios, and
outputs a first multi-classification relevance score associated with the customer, wherein the first multi-classification relevance score comprises a first score corresponding to the first scenario, and wherein each score in the multi-classification relevance score is calculated as a distance from a corresponding numerical representation to a centroid of a corresponding cluster;
and
in response to determining that the first score meets a threshold value, set the visual indicator to show that the likelihood is high that the alert corresponding to the customer is false positive.

5 more elements

A method of training a machine learning (ML) model for enhanced triaging of alerts generated for customer service advisors that are servicing customers to reduce false positive alerts, wherein interactions between a customer and one or more customer service advisors are recorded in textual notes, and wherein an alert for servicing the customer indicates a first scenario from among an enumerated list of scenarios, method comprising:

filtering keywords in the textual notes to remove a first set of keywords that fail to correspond to any specific scenario among the enumerated list of scenarios;
after the filtering of the keywords, transforming the textual notes into a text-frequency matrix, wherein the first set of keywords are omitted from the text-frequency matrix, and wherein each row of the text-frequency matrix represents a different textual note;
reducing a dimension of the text-frequency matrix, wherein the text-frequency matrix is outputted as numerical representations;
k-means clustering on the numerical representations outputted by the text-frequency matrix to produce a plurality of clusters, each cluster corresponding to a single scenario in the enumerated list of scenarios;
and
for each textual note, outputting, by the ML model, a multi-classification relevance score that includes a quantity of scores matching a quantity of clusters in the plurality of clusters, and wherein each cluster corresponds to a scenario from the enumerated list of scenarios, and wherein each score in the multi-classification relevance score is calculated as a distance from the corresponding numerical representation to a centroid of a corresponding cluster and indicates a likelihood that the customer was previously notified of the corresponding scenario for that corresponding cluster.
wherein a computer system that implements the ML model is configured to, in response to finding that the score in the multi-classification relevance score corresponding to the first scenario satisfies a threshold value, suppressing generation of the alert for servicing the customer about the first scenario as a false positive alert.

4 more elements

A system configured to triage alerts generated for customer service advisors to reduce false positive alerts, the system comprising:

a RMS notes data store configured to store textual notes about interactions between the customer service advisors and a customer;
a text-frequency matrix created by concatenating the textual notes into an unstructured conversation thread associated with the customer, wherein each row of the text-frequency matrix represents a different textual note;
a module configured for dimensionality reduction of the text-frequency matrix using principal component analysis (PCA);
a machine learning (ML) model configured to output a multi-classification relevance score associated with the customer using the dimensionally reduced text-frequency matrix and ML soft clustering, wherein each score of the multi-classification relevance score corresponds to a unique scenario in an enumerated list of scenarios;
a graphical user interface (GUI) configured to generate an alert for the customer service advisors, the alert being for the customer with a likelihood of satisfying a scenario from among the enumerated list of scenarios;
and
an artificial intelligence AI-bot that is configured to execute the ML model using one or more processors configured to:
determine that a first score in the multi-classification relevance score outputted by the ML model satisfies a threshold value, wherein the first score corresponds to a first scenario in the enumerated list of scenarios;
and
suppress, from the GUI, generation of the alert for servicing the customer about the first scenario, as a false positive alert.

7 more elements

Dependencies are inferred from the claim text.