Near-real-time continual 3d registration of objects in 2d x-ray images

A method of assisting in musculoskeletal surgery, the method:

• receiving a first X-ray image being a projection image of an at least partially visible object;
• receiving a model of the object and apply the model to localize the object in the first X-ray image;
• identifying a point in the first X-ray image, wherein a 3D position of the point relative to the object is known;
• receiving a second X-ray image being a projection image of an at least partially visible tool together with the at least partially visible object;
• applying the model of the object to localize the object in the second X-ray image;
• defining a coordinate system in fixed relation to the tool;
• determining and identifying an axis in the coordinate system;
• and
• determining a 3D position and orientation of the tool relative to the object based on:
• (i) the second X-ray image;
• (ii) a 3D model of the tool;
• (iii) the localized object;
• (iv) knowledge that the 3D position of the point relative to the object is the same when generating the first X-ray image and when generating the second X-ray image;
• and
• (v) knowledge about the distance between the point and the axis.

A device for assisting in musculoskeletal surgery comprising:

• a processing unit configured to receive X-ray image data representing an X-ray projection image of an at least partially visible object, wherein the processing unit is programmed to:
• receive a first X-ray image being a projection image of an at least partially visible object,
• receive a model of the object and apply the model to localize the object in the first X-ray image,
• identify a point in the first X-ray image, wherein a 3D position of the point relative to the object is known,
• receive a second X-ray image being a projection image of an at least partially visible tool together with the at least partially visible object,
• apply the model of the object to localize the object in the second X-ray image,
• define a coordinate system in fixed relation to the tool,
• determine and identifying an axis in the coordinate system, and
• determine a 3D position and orientation of the tool relative to the object based on:
• (i) the second X-ray image;
• (ii) a 3D model of the tool;
• (iii) the localized object;
• (iv) knowledge that the 3D position of the point relative to the object is the same when generating the first X-ray image and when generating the second X-ray image;
• and
• (v) knowledge about the distance between the point and the axis.