Multi-tissue segmentation and deformation for breast cancer surgery

A non-transitory computer-readable medium, having stored thereon program instructions that, upon execution by a computing system, cause the computing system to perform operations comprising:

• obtaining 3D image data of breast tissue;
• applying a tumor localization procedure to the 3D image data, wherein applying the tumor localization procedure includes using a tumor localization neural network ensemble to predict 3D bounding boxes of tumor locations within the 3D image data;
• applying a tumor segmentation procedure to the 3D image data, wherein applying the tumor segmentation procedure includes using a tumor segmentation neural network to predict first probabilities that a first set of locations within the breast tissue contain tumor tissue;
• applying a multi-tissue segmentation procedure to the 3D image data, wherein applying the multi-tissue segmentation procedure includes using a tissue segmentation neural network ensemble to predict second probabilities that a second set of locations within the breast tissue contain one or more types of non-tumor tissue, and combining the second probabilities using a weighted average;
• and
• determining most likely tissue types for regions of the 3D image data based on the 3D bounding boxes, the first probabilities from the tumor segmentation procedure, the second probabilities as combined from the multi-tissue segmentation procedure, and anatomical feasibilities of the breast tissue.

A system comprising:

• one or more processors;
• and
• memory, containing program instructions that, upon execution by the one or more processors, cause the system to perform operations comprising:
• obtaining 3D image data of breast tissue;
• applying a tumor localization procedure to the 3D image data, wherein applying the tumor localization procedure includes using a tumor localization neural network ensemble to predict 3D bounding boxes of tumor locations within the 3D image data;
• applying a tumor segmentation procedure to the 3D image data, wherein applying the tumor segmentation procedure includes using a tumor segmentation neural network to predict first probabilities that a first set of locations within the breast tissue contain tumor tissue;
• applying a multi-tissue segmentation procedure to the 3D image data, wherein applying the multi-tissue segmentation procedure includes using a tissue segmentation neural network ensemble to predict second probabilities that a second set of locations within the breast tissue contain one or more types of non-tumor tissue, and combining the second probabilities using a weighted average;
• and
• determining most likely tissue types for regions of the 3D image data based on the 3D bounding boxes, the first probabilities from the tumor segmentation procedure, the second probabilities as combined from the multi-tissue segmentation procedure, and anatomical feasibilities of the breast tissue.