Extended reality registration method using virtual fiducials

A system for placement of a virtual medical overlay on a physical body part using virtual fiducial markers, comprising:

• at least one memory;
• and
• at least one hardware processor of an extended reality device, wherein the at least one hardware processor is coupled with the at least one memory and configured, individually or in combination, to:
• retrieve, from the at least one memory, a three-dimensional virtual model of a body part, wherein the three-dimensional virtual model is tagged with a first plurality of virtual fiducial markers each indicative of a respective anatomical landmark on the body part;
• capture at least one image of the body part in a physical world;
• generate a request to mark, using the at least one image, a second plurality of virtual fiducial markers corresponding to anatomical landmarks indicated by the first plurality of virtual fiducial markers;
• in response to receiving a selection of the second plurality of virtual fiducial markers based on the request, generate a third plurality of virtual fiducial markers with corrected positions of the second plurality of virtual fiducial markers;
• and
• generate a virtual overlay of the three-dimensional virtual model on the body part captured by the extended reality device in real-time by performing registration on the first plurality of virtual fiducial markers and the third plurality of virtual fiducial markers.

A non-transitory computer readable medium storing thereon computer executable instructions for placement of a virtual medical overlay on a physical body part using virtual fiducial markers, including instructions for:

• retrieving, from memory by an extended reality device, a three-dimensional virtual model of a body part, wherein the three-dimensional virtual model is tagged with a first plurality of virtual fiducial markers each indicative of a respective anatomical landmark on the body part;
• capturing, by the extended reality device, at least one image of the body part in a physical world;
• generating a request to mark, using the at least one image, a second plurality of virtual fiducial markers corresponding to anatomical landmarks indicated by the first plurality of virtual fiducial markers;
• in response to receiving a selection of the second plurality of virtual fiducial markers based on the request, generating a third plurality of virtual fiducial markers with corrected positions of the second plurality of virtual fiducial markers;
• and
• generating a virtual overlay of the three-dimensional virtual model on the body part captured by the extended reality device in real-time by performing registration on the first plurality of virtual fiducial markers and the third plurality of virtual fiducial markers.