System, method and computer program product for assessing the capabilities of a conversation agent via black box testing
AssigneeInternational Business Machines Corporation
InventorsAlan BRAZ, Heloisa Caroline De Souza Pereira Candello, Claudio Santos Pinhanez, Marisa Affonso Vasconcelos
Life of this patent
Estimating the term…
Abstract
A conversational agent capability assessment method, system, and computer program product, includes assessing a performance, a personality and a cognitive trait of a conversational agent based on natural written language by a second conversational agent by combining an analysis of a plurality of metrics that each compare a metric from the conversational agent with a metric from the second conversational agent and producing a report detailing the performance, the personality, and the cognitive trait of the conversational agent
Claims20
A computer-implemented conversational agent capability assessment method, the method comprising:
- assessing a performance, a personality and a cognitive trait of a conversational agent based on natural written language by a second conversational agent by combining an analysis of a plurality of metrics that each compare a metric from the conversational agent with a metric from the second conversational agent;
- and
- producing a report detailing the performance, the personality, and the cognitive trait of the conversational agent.
A computer program product for conversational agent capability assessment, the computer program product comprising a computer-readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to perform:
- assessing a performance, a personality and a cognitive trait of a conversational agent based on natural written language by a second conversational agent by combining an analysis of a plurality of metrics that each compare a metric from the conversational agent with a metric from the second conversational agent;
- and
- producing a report detailing the performance, the personality, and the cognitive trait of the conversational agent.
A conversational agent capability assessment system, said system comprising:
- a processor;
- and
- a memory, the memory storing instructions to cause the processor to perform:
- assessing a performance, a personality and a cognitive trait of a conversational agent based on natural written language by a second conversational agent by combining an analysis of a plurality of metrics that each compare a metric from the conversational agent with a metric from the second conversational agent;
- and
- producing a report detailing the performance, the personality, and the cognitive trait of the conversational agent.
Dependencies are inferred from the claim text.