System and methods for monitoring trained machine learning models in a healthcare context

A system comprising:

• a first device located at a deployment site, wherein the first device comprises:
• a user input device;
• a first non-transitory memory including a trained machine learning model, and instructions;
• and
• a first processor, wherein, when executing the instructions, the first processor causes the first device to:
• extract characteristics of input data fed into the trained machine learning model, to produce an input data characterization, wherein the input data includes a medical image;
• extract characteristics of performance of the trained machine learning model during mapping of the medical image to an output, to produce a model performance characterization;
• extract characteristics of user feedback received via the user input device based on the output of the trained machine learning model, to produce a user feedback characterization;
• and
• transmit the input data characterization, the model performance characterization, and the user feedback characterization, to a second device;
• the second device located remotely from the first device, wherein the first device and the second device are communicatively coupled, and wherein the second device comprises:
• a second non-transitory memory including instructions;
• and
• a second processor, wherein, when executing the instructions, the second processor causes the second device to:
• receive the input data characterization, the model performance characterization, and the user feedback characterization, from the first device;
• determine an input data deviation by comparing the input data characterization against a plurality of previously determined input data characterizations;
• determine a model performance deviation by comparing the model performance characterization against a plurality of previously determined model performance characterizations;
• determine a user feedback deviation by comparing the user feedback characterization against a plurality of previously determined user feedback characterizations;
• and
• respond to one or more of the input data deviation exceeding an input data deviation threshold, the model performance deviation exceeding a model performance deviation threshold, and the user feedback deviation exceeding a user feedback deviation threshold by transmitting an alert to a user device.

A method for monitoring performance of a trained machine learning model, comprising:

• responding to a medical image of an imaging subject being input into the trained machine learning model by:
• extracting characteristics of the medical image of the imaging subject to produce an input data characterization, wherein the input data characterization excludes individual pixel or voxel intensity values and individual pixel or voxel locations;
• extracting characteristics of performance of the trained machine learning model during mapping of the medical image to an output, to produce a model performance characterization;
• determining an input data deviation by comparing the input data characterization against a plurality of previously determined input data characterizations;
• determining a model performance deviation by comparing the model performance characterization against a plurality of previously determined model performance characterizations;
• and
• responding to one or more of the input data deviation exceeding an input data deviation threshold, and the model performance deviation exceeding a model performance deviation threshold by transmitting an alert to a user device.