Runtime control of automation accuracy using adjustable thresholds

A computer-implemented method for maintaining a target accuracy level, comprising:

• receiving the target accuracy level;
• computing one or more thresholds comprising an ongoing adjustable automation threshold for a plurality of categories based on the target accuracy level;
• receiving data and generating a classification score for the plurality of categories with respect to the data based on a category knowledgebase;
• detecting that a classification score for a category with a higher classification score than other categories of the plurality of categories exceeds the ongoing adjustable automation threshold;
• automatically performing an action in response to the data based on the category with a higher classification score than the other categories of the plurality of categories;
• generating a suggestion list based on categories with higher classification scores than other categories for selection and feedback;
• monitoring outgoing actions including the action, the suggestion list, and corresponding received feedback to generate a historical performance dataset;
• determining an actual accuracy level based on the historical performance dataset;
• and
• adjusting, by a processor, the ongoing adjustable automation threshold based on the actual accuracy level.

A computer program product for maintaining a target accuracy level, the computer program product comprising a computer-readable storage medium having program code embodied therewith, the program code executable by a processor to cause the processor to:

• receive the target accuracy level;
• compute one or more thresholds comprising an ongoing adjustable automation threshold for a plurality of categories based on the target accuracy level;
• receive data and generate a classification score for the plurality of categories with respect to the data based on a category knowledgebase;
• detect that a classification score for a category with a higher classification score than other categories of the plurality of categories exceeds the ongoing adjustable automation threshold;
• automatically perform an action in response to the data based on a category with the higher classification score than the other categories of the plurality of categories;
• generate a suggestion list based on categories with higher classification scores than other categories for selection and feedback;
• monitor the action, the suggestion list, and corresponding received feedback to generate a historical performance dataset;
• determine an actual accuracy level based on the historical performance dataset;
• and
• lower the ongoing adjustable automation threshold in response to detecting that the actual accuracy level is above the target accuracy level.