Techniques for comparing image contours of different human participants using automated tool

A computer-implemented method of comparing image contouring by different human participants without requiring that the different human participants contour a shared image dataset, the method comprising:

• receiving a first image dataset collection, including one or more first image datasets of one or more human or animal subjects produced by an imaging modality, and including first contouring data generated by a first human participant on the one or more first image datasets;
• receiving a second image dataset collection, including one or more second image datasets of one or more human or animal subjects produced by the imaging modality, including second contouring data generated by a second human participant on the one or more second image datasets, the one or more second image datasets not overlapping with images in the one or more first image datasets;
• auto-contouring the first one or more first image datasets and the second one or more image datasets, using a computer-implemented autosegmentation engine to generate participant-specific computational reference contours third contouring data generated by the autosegmentation engine without requiring human contouring and without requiring identical or overlapping first and second image datasets, wherein the computational reference contours for each participant are generated from a participant's respective image datasets;
• comparing contouring by the first human participant to contouring by the second human participant by:
• comparing the first contouring data generated by the first human participant to the participant-specific computational reference contours of the third contouring data for the first human participant generated by the autosegmentation engine to generate first comparison data:
• comparing the second contouring data generated by the second human participant to the participant-specific computational reference contours of the third contouring data for the second human participant generated by the autosegmentation engine to generate second comparison data;
• and
• performing a transitive analysis comparison of the first human participant to the second human participant by separately analyzing the first comparison data and the second comparison data relative to a common reference provided by the third contouring data generated by the autosegmentation engine;
• generating a quality metric based on one or more systemic differences between contouring by the first and second human participants based on the transitive analysis;
• and
• displaying, on a user interface, the quality metric of the one or more systemic differences, wherein the quality metric includes visually distinguishable representations of missing, extra, and common volumes.

A computer-readable storage device comprising instructions, that when executed by at least one processor, configure the at least one processor to:

• receive a first image dataset collection, including one or more first image datasets of one or more human or animal subjects produced by an imaging modality, and including first contouring data generated by a first human participant on the one or more first image datasets;
• receive a second image dataset collection, including one or more second image datasets of one or more human or animal subjects produced by the imaging modality, including second contouring data generated by a second human participant on the one or more second image datasets, the one or more second image datasets not overlapping with images in the one or more first image datasets;
• auto-contour the first one or more first image datasets and the second one or more image datasets, using a computer-implemented autosegmentation engine to generate participant-specific computational reference contours third contouring data generated by the autosegmentation engine without requiring human contouring and without requiring identical or overlapping first and second image datasets, wherein the computational reference contours for each participant are generated from a participant's respective image datasets;
• compare contouring by the first human participant to contouring by the second human participant by:
• comparing the first contouring data generated by the first human participant to the participant-specific computational reference contours of the third contouring data for the first human participant generated by the autosegmentation engine to generate first comparison data:
• comparing the second contouring data generated by the second human participant to the participant-specific computational reference contours of the third contouring data for the second human participant generated by the autosegmentation engine to generate second comparison data;
• and
• performing a transitive analysis comparison of the first human participant to the second human participant by separately analyze the first comparison data and the second comparison data relative to a common reference provided by the third contouring data generated by the autosegmentation engine;
• generate an indication of one or more systemic differences between contouring by the first and second human participants based on the transitive analysis;
• and
• display, on a user interface, the indication of the one or more systemic differences, wherein the indication includes visually distinguishable representations of missing, extra, and common volumes.

A system for comparing image contouring by different human participants without requiring that the different human participants contour a shared image dataset, the system comprising:

• at least one processor;
• and
• computer-readable storage device comprising instructions, that when executed by at least one processor, configure the at least one processor to:
• receive a first image dataset collection, including one or more first image datasets of one or more human or animal subjects produced by an imaging modality, and including first contouring data generated by a first human participant on the one or more first image datasets;
• receive a second image dataset collection, including one or more second image datasets of one or more human or animal subjects produced by the imaging modality, including second contouring data generated by a second human participant on the one or more second image datasets, the one or more second image datasets not overlapping with images in the one or more first image datasets;
• auto-contour the first one or more first image datasets and the second one or more image datasets, using a computer-implemented autosegmentation engine to generate participant-specific computational reference contours third contouring data generated by the autosegmentation engine without requiring human contouring and without requiring identical or overlapping first and second image datasets, wherein the computational reference contours for each participant are generated from a participant's respective image datasets;
• compare contouring by the first human participant to contouring by the second human participant by:
• comparing the first contouring data generated by the first human participant to the participant-specific computational reference contours of the third contouring data for the first human participant generated by the autosegmentation engine to generate first comparison data:
• comparing the second contouring data generated by the second human participant to the participant-specific computational reference contours of the third contouring data for the second human participant generated by the autosegmentation engine to generate second comparison data;
• and
• performing a transitive analysis comparison of the first human participant to the second human participant by separately analyze the first comparison data and the second comparison data relative to a common reference provided by the third contouring data generated by the autosegmentation engine;
• generate an indication of one or more systemic differences between contouring by the first and second human participants based on the transitive analysis;
• and
• display, on a user interface, the indication of the one or more systemic differences, wherein the indication includes visually distinguishable representations of missing, extra, and common volumes.