Multi-class image segmentation with w-net architecture
AssigneesSmith & Nephew, Inc., Smith & Nephew Orthopaedics AG, Smith & Nephew Asia Pacific Pte. Limited
InventorsDarren J. WILSON, Branislav JARAMAZ
Life of this patent
Estimating the term…
Abstract
A system for markerless registration and tracking is disclosed. The system includes an imaging sensor configured to capture both RGB images and depth maps of environment. The system can be configured to receive an RGB image and associated depth information from the imaging sensor, segment the RGB image, using a deep learning network, by classifying each pixel as belonging to one of the group of proximal tibia, distal femur, patella, or non-boney material of the knee, and determine a loss based on a comparison between the predicted segmentation mask and a ground-truth mask. The ground-truth mask may be generated based on the depth map captured by the imaging sensor.
Claims20
A system for intraoperative multi-class segmentation of a patient's proximal tibia, distal femur, and patella, comprising:
- an imaging sensor configured to capture and RGB frame and associated depth data;
- a processor;
- and
- a non-transitory, processor-readable storage medium in communication with the processor, wherein the non-transitory, processor-readable storage medium contains one or more programming instructions that, when executed, cause the processor to:
- receive the RGB frame and the associated depth data from the imaging sensor,
- segment the RGB frame into a predicted segmentation mask, using a deep learning network supported by object detection, by classifying each pixel as belonging to one of the group of proximal tibia, distal femur, patella, or non-boney material of the knee, and
- determine a loss based on a comparison between the predicted segmentation mask and a ground-truth mask.
A method of determining a pose of a patient anatomy, the method comprising:
- receiving imagery from an imaging sensor, wherein the imaging sensor produces RGB images and associated depth data;
- segmenting the imagery based on the patient anatomy visible in the imagery, wherein the segmenting comprises classifying any of a femur, tibia, or patella present in the imagery;
- generating a 3D point cloud based on the depth data;
- constructing a 3D surface of the patient anatomy by applying the segmentation to the 3D point cloud;
- and
- determining a pose of the patient anatomy by aligning the 3D surface of the patient anatomy with at least one of a 3D pre-operative model of the patient or an atlas model.
Dependencies are inferred from the claim text.