System and methods for characterizing and monitoring inter-anatomical reference relationships in medical imaging

A system comprising:

• a first device located at a clinical site, wherein the first device comprises:
• a first non-transitory memory including a deployed machine learning model and instructions;
• and
• a first processor, wherein, when executing the instructions, the first processor causes the first device to:
• receive a medical image;
• detect a plurality of anatomical landmarks in the medical image using the deployed machine learning model;
• determine spatial relationships between pairs of the detected anatomical landmarks;
• and
• transmit the determined spatial relationships to a second device without transmitting the medical image;
• the second device located remotely from the first device, wherein the first device and the second device are communicatively coupled, and wherein the second device comprises:
• a second non-transitory memory including instructions;
• and
• a second processor, wherein, when executing the instructions, the second processor causes the second device to:
• receive the determined spatial relationships from the first device;
• compare the determined spatial relationships against a plurality of previously determined spatial relationships to identify outlier relationships;
• generate a performance characterization of the deployed machine learning model based on the identified outlier relationships;
• and
• respond to the performance characterization indicating a deviation from expected model performance by transmitting an alert to a user device.

A method for monitoring performance of a deployed machine learning model, comprising:

• receiving, at a remote monitoring site, a plurality of spatial interrelationship measurements between anatomical landmarks detected by the deployed machine learning model in a medical image at a deployment site, wherein the spatial interrelationship measurements exclude the medical image and patient identifying information, and wherein the anatomical landmarks comprise one or more geometric objects including organ masks delineating boundaries of anatomical organs, segmentation masks identifying anatomical structures, landmark masks indicating positions of anatomical reference points, planes derived from the anatomical reference points, and bounding boxes encompassing regions of interest;
• accessing a reference dataset comprising spatial interrelationship measurements between corresponding anatomical landmarks detected in a plurality of training images;
• for each type of spatial interrelationship measurement between pairs of anatomical landmarks:
• determining statistical measures from spatial interrelationship measurements from the reference dataset of the type;
• determining an upper threshold and a lower threshold based on the statistical measures determined from the reference dataset;
• comparing each received spatial interrelationship measurement against the upper threshold and lower threshold determined for the type of spatial interrelationship measurement;
• determining a frequency of outlier occurrences for each anatomical landmark by counting a number of times the anatomical landmark appears in spatial interrelationship measurements outside the upper threshold and lower bound threshold;
• identifying one or more anomalous anatomical landmarks based on the frequency of outlier occurrences exceeding a predetermined frequency threshold;
• and
• transmitting an alert identifying the one or more anomalous anatomical landmarks to the deployment site.