Adaptive framework for automated artificial intelligence process

A document processing computing system comprising:

• a confidence calibration service comprising an artificial intelligence (AI) model for determining a confidence level that a document, when processed automatically, will meet a pre-defined target of processing accuracy;
• a routing orchestrator for adopting automatic processing of the document if the determined confidence level is equal to or above a confidence threshold;
• a quality assurance workbench for a manual verification of M automatically processed documents out of N automatically processed documents, wherein M<N;
• and
• a threshold manager for adjusting the confidence threshold depending on whether an incorrectly processed document has been identified among the verified M documents.

An automatic decision-making computing system comprising:

• a confidence calibration service comprising an artificial intelligence (AI) model for determining a confidence level that an AI action on an object will meet a pre-defined target of action accuracy;
• a routing orchestrator for adopting the AI action if the determined confidence level is equal to or above a confidence threshold;
• a sampling engine for selecting M out of N AI actions, wherein M <N;
• a quality assurance workbench for at least one of:
• a manual action if the determined confidence level for the AI action is below the confidence threshold;
• or a manual verification of the M out of the N AI actions;
• a metrics engine for estimating, upon processing a pool of objects either manually or via an AI action depending on the determined confidence level in relation to the confidence threshold, an error rate in the pool of processed objects based on the verification of all manually performed actions and the M AI actions, wherein the error rate is estimated based on a weighted average of an error rate of the verified manual actions, an error rate of the verified M AI actions, and an estimated error rate of non-verified N-M AI actions;
• and
• a threshold manager for adjusting the confidence threshold depending on whether an incorrect AI action has been identified among the verified M AI actions.