Detecting computer-generated hallucinations using progressive scope-of-analysis enlargement

A computing system for determining whether a target item contains a computer-generated artifact produced by a computer-implemented transformation system, comprising:

• an instruction data store for storing computer-readable instructions;
• a processing system for executing a computer program based on the computer-readable instructions in the data store, to perform operations including:
• receiving a source item, the source item having plural source parts;
• receiving the target item, the target item having plural target parts, the target item being produced by transforming the source item into the target item using the computer-implemented transformation system, the computer-implemented transformation system being a first machine-trained model that has been trained to produce tokens of target items that summarize tokens of source items;
• determining whether the target parts in the target item are supported by the source item using an analysis process, the analysis process including logic for progressively expanding a scope of source content to be considered when analyzing whether a particular target part of the target item is supported by the source item,
• wherein the determining uses a computer-implemented natural language interface (NLI) system, the NLI system being a second machine-trained model,
• wherein the second machine-trained model that implements the NLI system has been trained to transform tokens of an instance of target content and an instance of source content into a classification result that indicates whether the instance of target content is supported or unsupported by the instance of source content, the instance of target content containing at least one computer-generated artifact when determined to be unsupported by the instance of source content;
• assigning a label to the target item based on an outcome of the determining, the label indicating whether the target parts are supported by the source item;
• and
• automatically performing an action based on the label in response to concluding at least one part of the target item is not supported by the source item.

A non-transitory computer-readable storage medium for storing computer-readable instructions, wherein a processing system executes the computer-readable instructions to perform operations, the operations comprising each of:

• determining whether a particular target part of a target item is supported by any individual source part in a source item having plural source parts using a computer-implemented machine-trained natural language interface (NLI) system,
• the target item being produced by transforming the source item into the target item using a computer-implemented transformation system, the computer-implemented transformation system being a first machine-trained model that has been trained to produce tokens of target items that summarize tokens of source items;
• in response to determining that the particular target part is not supported by any source part, selecting a group of particular source parts that most closely match the particular target part, as determined by a computer-implemented similarity-assessment process, the group having a predetermined number of source parts;
• concatenating the particular source parts in an order that matches an order in which the particular source parts appear in the source item;
• determining whether the particular target part is supported by the group of particular source parts using the NLI system, the NLI system being a second machine-trained model;
• increasing a size of the group by one source part in response to concluding that the particular target part is unsupported by the group;
• and
• repeating the selecting, concatenating, determining whether the particular target part is supported by the group of particular source parts, and increasing until it is determined that the particular target part is supported, or until determining that a maximum size of the group of particular source parts has been reached and the particular target part remains unsupported and contains a computer-generated artifact,
• the operations further including, in response to determining that at least one target part is unsupported, automatically performing an action,
• wherein the second machine-trained model that implements the NLI system has been trained to transform tokens of an instance of target content and an instance of source content into a classification result that indicates whether the instance of target content is supported or unsupported by the instance of source content, the instance of target content containing at least one computer-generated artifact when determined to be unsupported by the instance of source content.