US 2026/0203498 A1Published application

Fallback analyzer for automated analysis of computerized conversational agent conversational data

AssigneeConversational AI Ltd, dba The CAI Company

InventorsDarren Ford, Allison Adams, Alice Kerly, Allan Osmond, Joe Yelland

Life of this patent

Estimating the term…

Loading patent…

Abstract

A system and method for automated qualitative analysis of computerized conversational agent (CCA) conversational data. In an embodiment, the system comprises a system configuration manager for establishing CCA characteristics and factors to be included in analyses, a data ingestion engine for ingesting conversations from a CCA, a conversation indicator extractor for identifying and classifying key aspects of conversations, a fallback analyzer for identifying the type and frequency of CCA failures, a topic modeling engine for identifying trends in conversations, and a CCA score generator for generating score assessments for qualitative aspects of the CCA's performance.

Claims20

Dependent claims

A system for automated analysis of computerized conversational agent (CCA) conversational data, comprising:

a computer system comprising a memory, a processor, and a data store residing on a non-volatile data storage device;
transcript data stored in the data store, the transcript data comprising text of one or more conversations between a computerized conversational agent (CCA) and a user, each conversation comprising a plurality of messages, each message being labeled as having been authored either by the CCA or by the user;
and
a fallback analyzer comprising a first plurality of programming instructions stored in the memory which, when operating on the processor, causes the computer system to:
retrieve from the transcript data messages associated with a fallback indicator;
and
for each retrieved message:
extract fallback content from the conversation, the fallback content comprising:
an agent message indicated as containing a fallback message;
a customer message preceding the fallback message;
and
an agent message that elicited the customer message preceding the fallback message;
send the fallback content to an artificial intelligence assistant (AI assistant) with instructions to the AI assistant to assign a fallback type for the fallback message as either intent-based or prompt-based;
and
store the assigned fallback type in the data store.

12 more elements

A method for automated analysis of computerized conversational agent (CCA) conversational data, comprising the steps of:

storing transcript data a data store of a computer system comprising a memory, a processor, and the data store residing on a non-volatile data storage device, the transcript data comprising text of one or more conversations between a computerized conversational agent (CCA) and a user, each conversation comprising a plurality of messages, each message being labeled as having been authored either by the CCA or by the user;
using a fallback analyzer operating on the computer system to perform the steps of:
retrieving from the transcript data messages associated with a fallback indicator;
and
for each retrieved message:
extracting fallback content from the conversation, the fallback content comprising:
an agent message indicated as containing a fallback message;
a customer message preceding the fallback message;
and
an agent message that elicited the customer message preceding the fallback message;
sending the fallback content to an artificial intelligence assistant (AI assistant) with instructions to the AI assistant to assign a fallback type for the fallback message as either intent-based or prompt-based;
and
storing the assigned fallback type in the data store.

10 more elements

Dependencies are inferred from the claim text.