Hybrid hardware and computer vision-based tracking system and method
AssigneesChildren's National Medical Center, Igi Technologies
InventorsRaj Shekhar, William Plishker, Xinyang Liu
Life of this patent
Estimating the term…
Abstract
The present disclosure relates to a tracking system for augmented reality in a clinical setting. Specifically, the present disclosure relates to an approach for combining hardware-based tracking and computer vision-based tracking in order to accurately overlay a projected image onto a video image.
Claims20
A tracking system for augmented reality, the tracking system comprising:
- an imaging modality configured to obtain a medical image;
- a display configured to display the medical image;
- and
- an image control device including processing circuitry configured to
- acquire spatial data of a real object, the real object being a medical surgical tool observed in the medical image of the imaging modality, via a first tracking method,
- determine a projection matrix based upon the acquired spatial data of the real object, the acquired spatial data corresponding to points observed on the real object,
- project a virtual object onto the medical image of the imaging modality, the virtual object being a projection of a shape model of the real object according to the projection matrix,
- calculate a correction matrix based on the projection of the virtual object,
- update the projection of the virtual object according to the correction matrix, and
- modify the display of the medical image based on the updated projection, the medical image including an ultrasound image overlayed on a camera image.
A method for augmented reality, comprising:
- acquiring, via processing circuitry, spatial data of a real object, the real object being a medical surgical tool observed in a medical image of an imaging modality via a first tracking method;
- determining, via the processing circuitry, a projection matrix based upon the acquired spatial data of the real object, the acquired spatial data corresponding to points observed on the real object;
- projecting, via the processing circuitry, a virtual object onto the medical image of the imaging modality, the virtual object being a projection of a shape model of the real object according to the projection matrix;
- calculating, via the processing circuitry, a correction matrix based on the projection of the virtual object;
- updating, via the processing circuitry, the projection of the virtual object according to the correction matrix;
- and
- displaying, via the processing circuitry, the medical image, wherein the displayed medical image includes an ultrasound image overlayed on a camera image and wherein the displayed medical image is modified based upon the updated projection.
A non-transitory computer-readable storage medium storing computer-readable instructions that, when executed by a computer, cause the computer to perform a method of augmented reality, comprising:
- acquiring spatial data of a real object, the real object being a medical surgical tool observed in a medical image of an imaging modality, via a first tracking method;
- determining a projection matrix based upon the acquired spatial data of the real object the acquired spatial data corresponding to points observed on the real object;
- projecting a virtual object onto the medical image of the imaging modality, the virtual object being a projection of a shape model of the real object according to the projection matrix;
- calculating a correction matrix based on the projection of the virtual object;
- updating the projection of the virtual object according to the correction matrix;
- and
- displaying the medical image, wherein the displayed medical image is modified based upon the updated projection and wherein the medical image includes an ultrasound image overlayed on a camera image.
Dependencies are inferred from the claim text.