Evaluation framework for intent authoring processes

A system for evaluating intent authoring processes, comprising:

• a processor executing instructions stored in a memory device;
• wherein the processor:
• receives results of a simulated intent labeling effort of a dataset comprising utterances of interactive dialog sessions between agents and clients for a given product or service, wherein a plurality of snapshots of an intent state are generated during the simulated intent labeling effort;
• computes figures of merits for respective algorithms used to perform the simulated intent labeling effort;
• evaluates each of the respective algorithms according to the computed figures of merits;
• and
• implements one of the respective algorithms for labeling intents of a remaining corpus of the synthesized dataset according to parameters evaluated in the computed figures of merits.

A computer program product for evaluating intent authoring processes, the computer program product comprising a non-transitory computer-readable storage medium having computer-readable program code portions stored therein, the computer-readable program code portions comprising:

• an executable portion that receives results of a simulated intent labeling effort of a dataset comprising utterances of interactive dialog sessions between agents and clients for a given product or service, wherein a plurality of snapshots of an intent state are generated during the simulated intent labeling effort;
• an executable portion that computes figures of merits for respective algorithms used to perform the simulated intent labeling effort;
• an executable portion that evaluates each of the respective algorithms according to the computed figures of merits;
• and
• an executable portion that implements one of the respective algorithms for labeling intents of a remaining corpus of the synthesized dataset according to parameters evaluated in the computed figures of merits.