Methods and systems for monitoring distributed data-driven models

A system for continuously monitoring a data-driven model configured to perform computer-assisted medical diagnostic tasks in a plurality of sites by generating, in each site of said plurality of sites, output data based on site-specific medical image input data, said system comprising:

• at least one processing device in each site that is configured to implement:
• a plurality of variants of the data-driven model deployed in a respective site of said plurality of sites, each variant of said plurality of variants being used in one of a plurality of states including a first state wherein the medical diagnostic output data of said each variant is included in computing a result of a task and a second state wherein the medical diagnostic output data of said each variant is excluded from computing the result of the task, and
• a supervision module in a respective site of said plurality of sites for monitoring said plurality of variants deployed in said respective site, the supervision module being configured:
• to compute the task result based on the medical diagnostic output data generated by at least two variants of said plurality of variants being used in the first state;
• to monitor the performance of the variants used in the first state and in the second state in accordance with at least one rule to detect performance improvement or degradation, based on the medical diagnostic output data generated by the variants being used in the first state, and by the variants being used in the second state, and
• to change the use state of a variant from one state to another of said plurality of states based upon the performance monitoring, so as to retain or move to the first state those variants that meet a condition defined by said at least one rule.