# US 10,117,721 B2 · Real-time surgical reference guides and methods for surgical applications > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,117,721 B2 - Document ID: US10117721B2 - Status: Granted - Kind: B2 - Application number: 13/024,948 - Filed: 2011-02-10 - Published: 2011-06-30 - Granted: 2018-11-06 - Earliest claimed: 2008-10-10 - Assignees: TrueVision Systems, Inc. - Applicants: Ashok Burton Tripathi; Michael A. Weissman - Inventors: Ashok Burton Tripathi; Michael A. Weissman - Web page: https://patentgrep.com/patent/US10117721B2 ## CPC - A61B 90/37: {Surgical systems with images on a monitor during operation} - A61B 34/25: {User interfaces for surgical systems} - A61B 90/39: {Markers, e.g. radio-opaque or breast lesions markers} - A61F 2/0059: {Cosmetic or alloplastic implants (A61F2/10 - A61F2/16, A61F2/28 take precedence)} - A61F 2/12: Mammary prostheses - A61F 9/007: Methods or devices for eye surgery - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - A61F: FILTERS IMPLANTABLE INTO BLOOD VESSELS; PROSTHESES; DEVICES PROVIDING PATENCY TO, OR PREVENTING COLLAPSING OF, TUBULAR STRUCTURES OF THE BODY, e.g. STENTS; ORTHOPAEDIC, NURSING OR CONTRACEPTIVE DEVICES; FOMENTATION; TREATMENT OR PROTECTION OF EYES OR EARS; BANDAGES, DRESSINGS OR ABSORBENT PADS; FIRST-AID KITS (dental prosthetics A61C) ## Abstract Apparatus and methods are described which guide a surgeon in performing a reconstructive or cosmetic procedure. The apparatus and methods utilize three dimensional presentations of the target surgical site incorporating one or more virtual surgical guides which help attain proper alignment and orientation of the post surgical outcome. ## Claims ### Claim 1 An apparatus for guiding a surgical reconstructive or cosmetic procedure on a patient, said apparatus comprising: a real-time, multidimensional visualization module including a photosensor and an image processor for producing real-time multidimensional visualizations of a portion of a target surgical site, the portion including an identifiable anatomical feature of the patient; a data processor communicatively coupled to the real-time, multidimensional visualization module and configured to generate in real-time: (i) at a first instant of time a first virtual surgical guide for display in conjunction with a first real-time multidimensional visualization of the portion of the target surgical site such that the first virtual surgical guide is aligned with respect to the identifiable anatomical feature, and (ii) at subsequent instances of time after the first instance of time subsequent virtual surgical guides for display in conjunction with respective subsequent real-time multidimensional visualizations of the portion of the target surgical site such that the subsequent virtual surgical guides are aligned with respect to the identifiable anatomical feature; and a display for presenting the real-time multidimensional visualizations in conjunction with the virtual surgical guides including the first real-time multidimensional visualization with the first virtual surgical guide and the subsequent real-time multidimensional visualizations with the respective subsequent virtual surgical guides, wherein the data processor is configured to: receive, from a user, an indication that (a) a first portion of the subsequent virtual surgical guides is to remain fixed regardless of movement of the identifiable anatomical feature, and (b) a second portion of the subsequent virtual surgical guides is to track movement of the identifiable anatomical feature, determine movement of the identifiable anatomical feature, using image processing, by determining a first location of the identifiable anatomical feature within the first real-time multidimensional visualization, determining subsequent locations of the identifiable anatomical feature displayed within the respective subsequent real-time multidimensional visualizations, and determining a difference between the first location and the subsequent locations, generate the subsequent virtual surgical guides in real time after the determined movement of the identifiable anatomical feature, and automatically position the subsequent virtual surgical guides such that the first portion of the subsequent virtual surgical guides remains fixed independent of the determined movement of identifiable anatomical feature while the second portion of the subsequent virtual surgical guides is moved by the determined difference so as to be in the same position as the first virtual surgical guide relative to the identifiable anatomical feature. ### Claim 2 (depends on 1) The apparatus according to claim 1, further comprising, a user control input in communication with the data processor for adjusting at least one of the first virtual surgical guide, the first portion of the subsequent virtual surgical guides, and the second portion of the subsequent virtual surgical guides, the user control input including at least one of a control panel, a keyboard, a mouse device, a joystick, a touchscreen device, a remote control, a voice activated device, a voice command device, a gesture recognition device, a position recognition device, and a foot switch. ### Claim 3 (depends on 1) The apparatus according to claim 1, wherein the data processor utilizes pre-operative patient data or intra-operative patient data to generate at least one of the first and subsequent virtual surgical guides. ### Claim 4 (depends on 3) The apparatus according to claim 3, wherein the pre-operative patient data or intra-operative patient data comprises at least one still image. ### Claim 5 (depends on 1) The apparatus according to claim 1, wherein at least one of the first and subsequent virtual surgical guides are drawn freehand. ### Claim 6 (depends on 1) The apparatus according to claim 1, wherein at least one of the first and subsequent virtual surgical guides are stereoscopic. ### Claim 7 (depends on 6) The apparatus according to claim 6, wherein the real-time multidimensional visualizations produced by the real-time, multidimensional visualization module are stereoscopic and high definition having a resolution of at least 1280 lines by 720 lines, and wherein at least one of the first and subsequent virtual surgical guides are presented by the display at a specified depth with respect to the respective stereoscopic and high definition first and subsequent real-time multidimensional visualizations. ### Claim 8 (depends on 1) The apparatus according to claim 1, wherein the apparatus is configured to guide the surgical reconstructive procedure or cosmetic procedure is by guiding a procedure selected from the group consisting of breast augmentation, lipoplasty, eyelid surgery, rhinoplasty, abdominoplasty, botulinum toxin injections, facelift, cheek implantation, dermal filler injections, transdermal treatments, heat treatments, UV treatments, broken nose repair, broken finger repair, ear defect reconstruction, skin reconstruction, birth defect reconstruction, and cleft palate repair. ### Claim 9 (depends on 1) The apparatus according to claim 1, wherein the apparatus is configured to produce the real-time multidimensional visualizations of the target surgical site producing the real-time multidimensional visualizations of a site selected from the group consisting of a nose, one or more eyes, one or more cheeks, a forehead, one or more ears, one or more breasts, an abdomen, one or more hands, one or more feet, one or more legs, one or more teeth, and combinations thereof. ### Claim 10 An apparatus for guiding a surgical reconstructive or cosmetic procedure on a patient, said apparatus comprising: a real-time, multidimensional visualization module including a photosensor and an image processor for producing real-time multidimensional visualizations of a portion of a target surgical site, the portion including an identifiable anatomical feature of the patient; a data processor communicatively coupled to the real-time, multidimensional visualization module and configured to generate in real-time: (i) at a first instant of time a virtual surgical guide for display in conjunction with a first real-time multidimensional visualization of the portion of the target surgical site such that the virtual surgical guide is aligned with respect to the identifiable anatomical feature, and (ii) at subsequent instances of time after the first instance of time the virtual surgical guide for display in conjunction with respective subsequent real-time multidimensional visualizations of the portion of the target surgical site such that the virtual surgical guide is aligned with respect to the identifiable anatomical feature; and a display for presenting the real-time multidimensional visualizations in conjunction with the virtual surgical guide, wherein the data processor is configured to: receive, from a user, an indication that (a) a first portion of the virtual surgical guide is to remain fixed regardless of motion of the identifiable anatomical feature, and (b) a second portion of the virtual surgical guide is to track motion of the identifiable anatomical feature, determine motion of the identifiable anatomical feature, using image processing, by identifying a first location of the identifiable anatomical feature within the first real-time multidimensional visualization, identifying at least one subsequent location of the identifiable anatomical feature displayed within the at least one respective subsequent real-time multidimensional visualization, and determining a difference between the first location and the subsequent locations, and adjust the virtual surgical guide such that the first portion of the virtual surgical guide remains fixed independent of the determined motion of identifiable anatomical feature while the second portion of the virtual surgical guide is moved by the determined difference so as to be in the same position relative to the identifiable anatomical feature after the determined motion. ### Claim 11 (depends on 1) The apparatus according to claim 1, wherein the first virtual surgical guide and the subsequent virtual surgical guides include at least one of a line, an arc, a point, a circle, an axis, a line/plane of symmetry, an area, a topographical contour map, a pictorial representation, a measurement, a dimension, a freehand marking, an alphanumeric marking, and boundaries of a volumetric zone. ### Claim 12 (depends on 1) The apparatus according to claim 1, wherein the movement occurs as a result of actions during the surgical reconstructive or cosmetic procedure. ### Claim 13 (depends on 1) The apparatus according to claim 1, wherein the identifiable anatomical feature includes at least one of a scar, a vascular pattern, a body part, a mole, tissue, a protrusion, a skin feature, a bone feature, an annotation, a reference marking, and combinations thereof. ### Claim 14 (depends on 1) The apparatus according to claim 1, wherein the display includes a head-mounted display. ### Claim 15 (depends on 10) The apparatus according to claim 10, further comprising a user control input in communication with the data processor and configured to enable a user to provide the indication that the first portion of the subsequent virtual surgical guide is to remain fixed. ### Claim 16 (depends on 15) The apparatus according to claim 15, wherein the user control input enables a user to lock, relative to the identifiable anatomical feature, at least one of a line, an arc, a point, a circle, an axis, a line/plane of symmetry, an area, a topographical contour map, a pictorial representation, a measurement, a dimension, a freehand marking, an alphanumeric marking, and boundaries of a volumetric zone of the virtual surgical guide. ### Claim 17 (depends on 10) The apparatus according to claim 10, wherein at least one of the virtual surgical guide, the first portion of the virtual surgical guide, and the second portion of the virtual surgical guide at least represents locations or dimensions of desired intraoperative incisions. ### Claim 18 (depends on 10) The apparatus according to claim 10, wherein at least one of the virtual surgical guide, the first portion of the virtual surgical guide, and the second portion of the virtual surgical guide at least represents locations or dimensions of anatomical features of the patient. ### Claim 19 (depends on 10) The apparatus according to claim 10, wherein at least one of the virtual surgical guide, the first portion of the virtual surgical guide, and the second portion of the virtual surgical guide includes a pictorial representation of the portion of the target surgical site. ### Claim 20 (depends on 10) The apparatus according to claim 10, wherein the real-time, multidimensional visualization module is configured to produce the real-time multidimensional visualizations for stereoscopic presentation by the display. ### Claim 21 (depends on 10) The apparatus according to claim 10, wherein the identifiable anatomical feature includes at least one of a scar, a vascular pattern, a body part, a mole, tissue, a protrusion, a skin feature, a bone feature, an annotation, a reference marking, and combinations thereof. ### Claim 22 (depends on 12) The apparatus according to claim 12, wherein the actions include at least one of: making one or more incisions; removing tissue; adding tissue or an implant; orienting an implant; injecting botulinum toxin; lipoplasty; inflammation of tissue; and manipulation of the patient during the performance of the actions thereof. ### Claim 23 (depends on 1) The apparatus according to claim 1, wherein the data processor is configured to: receive, in the data processor, from a user, a selection of the identifiable anatomical feature of the patient; receive, in the data processor from the user, an indication that the second portion of the subsequent virtual surgical guides is to move in relation to only the identifiable anatomical feature of the patient; and disregard, via the data processor, movement of other identifiable anatomical features of the patient such that the second portion of the subsequent virtual surgical guides is automatically positioned to track the determined movement of only the identifiable anatomical feature of the patient.