Augmented and mixed reality incorporating pathology results in surgical settings
OwnerMichael E. Ivan, Max Cacchione +1inventors
InventorsMichael E. Ivan, Max Cacchione, Adam McMahon
Life of this patent
Estimating the term…
Abstract
Apparatuses and methods for using augmented/mixed reality in surgical settings are provided, and, in particular, apparatuses and methods for intraoperative integration of augmented/mixed reality 3D models with near real-time pathology results. Real-time pathology results are generated using a machine learning analysis of biopsy pathology images in conjunction with a predictive model based on the patient's clinical and demographic data and radiographic imaging data. The apparatuses and methods can be used to provide more accurate intraoperative visualization, tracking, and determination of tumor margins to improve resection extent and any intraoperative or postoperative adjuvant therapy.
Claims20
A method for providing intraoperative surgery guidance, comprising:
- generating, with a processor of a computer system, a 3D model of an area of interest on a patient's body;
- based on the 3D model of the area of interest, providing a representation of the area of interest displayed on a head-mounted display device worn by a user, the representation of the area of interest displayed as an overlay over the user's view of the area of interest through the head-mounted display device during a surgical operation, the representation including pathology characteristics associated with the area of interest;
- detecting, with the processor during the surgical operation, a tissue removal and a section location associated with the tissue removal;
- associating the section location with a pathology result associated with the tissue removal in the 3D model of the area of interest to provide an updated 3D model of the area of interest during the surgical operation;
- and
- based on the pathology result, updating an indicator associated with the section location of the tissue removal in the representation of the area of interest displayed on the head-mounted display device to provide an updated representation of the area of interest during the surgical operation.
An apparatus for providing intraoperative and postoperative surgery guidance, comprising:
- a computer system operable to generate a 3D model of an area of interest on a patient's body;
- a head-mounted augmented/mixed reality display device to be worn by a user, for generating a representation of the area of interest based on the 3D model of the area of interest displayed as an overlay over a user's view of the area of interest during a surgical operation, wherein the representation includes pathology characteristics associated with the area of interest;
- and
- a central server of the computer system, the central server operable to:
- communicate, during the surgical operation, with the head-mounted augmented/mixed reality display device to detect a tissue removal and a section location associated with the tissue removal,
- associate the section location with a pathology result associated with the tissue removal in the 3D model of the area of interest to provide an updated 3D model of the area of interest during the surgical operation, and
- based on the pathology result, update an indicator associated with the section location of the tissue removal in the representation of the area of interest displayed on the head-mounted display device to provide an updated representation of the area of interest during the surgical operation.
Dependencies are inferred from the claim text.