Comparing Ordered Sets Generated by Different Data-Processing Engines

A computing system for updating data-processing functionality, comprising:

• a memory for storing machine-readable instructions;
• and
• a processing system that executes the machine-readable instructions to perform operations of:
• obtaining a first ordered set of first-set result items produced by a first data-processing engine in response to an input query;
• obtaining a second ordered set of second-set result items produced by a second data-processing engine in response to the same input query, the second data-processing engine serving as a reference engine that provides ground-truth result items;
• generating accuracy information by assessing accuracy of the first-set result items relative to the second-set result items,
• the generating being based on an assessment of item similarity between individual pairings formed from the first ordered set and the second ordered set, and based on a comparison of an order of the first-set result items and an order of the second-set result items;
• and
• producing output information that identifies deficiencies in the first data-processing engine based on the accuracy information, which enables modification of a configuration of the first data-processing engine to increase quality of result items produced by the first data-processing engine in a production stage of operation.

A computer-readable storage medium for storing computer-readable instructions, the computer-readable memory excluding propagated signals in transit, wherein a processing system executing the computer-readable instructions performs operations comprising:

• obtaining a first ordered set of first-set result items produced by a first data-processing engine in response to an input query;
• obtaining a second ordered set of second-set result items produced by a second data-processing engine in response to the same input query, the second data-processing engine serving as a reference engine that provides ground-truth result items;
• generating accuracy information by assessing accuracy of the first-set result items relative to the second-set result items,
• the generating being based on an assessment of item similarity between individual pairings formed from the first ordered set and the second ordered set, and based on a comparison of an order of the first-set result items and an order of the second-set result items;
• and
• producing output information that identifies deficiencies in the first data-processing engine based on the accuracy information, which enables modification of a configuration of the first data-processing engine to increase quality of result items produced by the first data-processing engine in a production stage of operation.